[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:44:46 dist-test-slave-d7lw krb5kdc[22335](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:44:46 dist-test-slave-d7lw krb5kdc[22335](info): set up 2 sockets
Sep 25 14:44:46 dist-test-slave-d7lw krb5kdc[22335](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 14:44:47.467491 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:44:47 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:44:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:44:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:44:47.578872 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33381
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:47.657650 22348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:47.657862 22348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:47.657897 22348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:47.658969 22348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:44:47.659030 22348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:47.659054 22348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:47.659081 22348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:44:47.659109 22348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:44:47.660506 22348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:47.660779 22348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:47.661037 22348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:47.666204 22353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:47.666254 22348 server_base.cc:1034] running on GCE node
W20260925 14:44:47.666245 22354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:47.666249 22356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:47.666697 22348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:47.667006 22348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:47.668159 22348 hybrid_clock.cc:648] HybridClock initialized: now 1790347487668139 us; error 29 us; skew 500 ppm
Sep 25 14:44:47 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347487, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:47.670715 22348 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:44:47.671068 22348 webserver.cc:460] Webserver started at http://127.21.205.254:38973/ using document root <none> and password file <none>
I20260925 14:44:47.671252 22348 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:47.671293 22348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:47.671386 22348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:47.672420 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "feef992aab8441a2a8d135881c221b12"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "62748527ccc7abdc8f7e34dcfae6f3f5"
server_key_iv: "816c9e00a338098fcd4421f776d27ff1"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.672722 22348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "feef992aab8441a2a8d135881c221b12"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "62748527ccc7abdc8f7e34dcfae6f3f5"
server_key_iv: "816c9e00a338098fcd4421f776d27ff1"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.673995 22348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:47.674651 22363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:47.674870 22348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:47.674944 22348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "feef992aab8441a2a8d135881c221b12"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "62748527ccc7abdc8f7e34dcfae6f3f5"
server_key_iv: "816c9e00a338098fcd4421f776d27ff1"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.674993 22348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:47.685557 22348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:47.687629 22348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:47.687753 22348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:47.691428 22348 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33381
I20260925 14:44:47.691478 22416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33381 every 8 connection(s)
I20260925 14:44:47.691874 22348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:44:47.692746 22417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:47.692881 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22348
I20260925 14:44:47.692997 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:44:47.693607 22327 external_mini_cluster.cc:1428] Setting key 485eaf0de6ed81f6a5541ef6d0ccd9df
I20260925 14:44:47.695459 22417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: Bootstrap starting.
I20260925 14:44:47.696110 22417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:47.696406 22417 log.cc:824] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:47.697106 22417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: No bootstrap required, opened a new log
I20260925 14:44:47.698911 22417 raft_consensus.cc:348] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } }
I20260925 14:44:47.699064 22417 raft_consensus.cc:374] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:47.699118 22417 raft_consensus.cc:729] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feef992aab8441a2a8d135881c221b12, State: Initialized, Role: FOLLOWER
I20260925 14:44:47.699295 22417 consensus_queue.cc:260] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } }
I20260925 14:44:47.699404 22417 raft_consensus.cc:388] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:44:47.699450 22417 raft_consensus.cc:482] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:44:47.699503 22417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:47.700179 22417 raft_consensus.cc:504] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } }
I20260925 14:44:47.700299 22417 leader_election.cc:302] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feef992aab8441a2a8d135881c221b12; no voters:
I20260925 14:44:47.700493 22417 leader_election.cc:288] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:44:47.700551 22422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:47.700682 22422 raft_consensus.cc:686] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [term 1 LEADER]: Becoming Leader. State: Replica: feef992aab8441a2a8d135881c221b12, State: Running, Role: LEADER
I20260925 14:44:47.700758 22422 consensus_queue.cc:237] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } }
I20260925 14:44:47.700820 22417 sys_catalog.cc:564] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:44:47.701088 22422 sys_catalog.cc:455] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader feef992aab8441a2a8d135881c221b12. Latest consensus state: current_term: 1 leader_uuid: "feef992aab8441a2a8d135881c221b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } } }
I20260925 14:44:47.701095 22424 sys_catalog.cc:455] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feef992aab8441a2a8d135881c221b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feef992aab8441a2a8d135881c221b12" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33381 } } }
I20260925 14:44:47.701177 22422 sys_catalog.cc:458] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:47.701184 22424 sys_catalog.cc:458] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12 [sys.catalog]: This master's current role is: LEADER
W20260925 14:44:47.702848 22438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:44:47.702903 22438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:44:47.702947 22428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:44:47.703097 22428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:44:47.704684 22428 catalog_manager.cc:1347] Generated new cluster ID: 4cc5a461ae4542bb98a933f30b33eee0
I20260925 14:44:47.704732 22428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:44:47 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:47.709120 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:47.694431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.21.205.254:33381)
0925 14:44:47.694581 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:47.694590 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:47.694855 (+ 265us) server_negotiation.cc:407] Connection header received
0925 14:44:47.696971 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:47.696977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:47.697019 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:47.697170 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:47.698527 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.699634 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.700993 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.701697 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.701832 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:47.705746 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:47.705750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:47.705750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:47.705761 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:47.707304 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:47.707723 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:47.707726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:47.707726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:47.707867 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:47.708130 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:47.708131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:47.708132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:47.708337 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:47.708496 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:47.708805 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 14:44:47.708933 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260925 14:44:47.726801 22428 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:44:47.727267 22428 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:44:47.733502 22428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feef992aab8441a2a8d135881c221b12: Generated new TSK 0
I20260925 14:44:47.733666 22428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:44:47.763018 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:47.868218 22444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:47.868436 22444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:47.868477 22444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:47.869630 22444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:47.869702 22444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:47.869750 22444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:44:47.871212 22444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:47.871474 22444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:47.871711 22444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:47.876617 22449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:47.876590 22452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:47.876598 22450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:47.877027 22444 server_base.cc:1034] running on GCE node
I20260925 14:44:47.877242 22444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:47.877547 22444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:47.878707 22444 hybrid_clock.cc:648] HybridClock initialized: now 1790347487878690 us; error 24 us; skew 500 ppm
Sep 25 14:44:47 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347487, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:47.881299 22444 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:44:47.881729 22444 webserver.cc:460] Webserver started at http://127.21.205.193:42507/ using document root <none> and password file <none>
I20260925 14:44:47.881953 22444 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:47.882020 22444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:47.882153 22444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:47.883160 22444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "485b9a233b9541eb8dbb149889b20c98"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "4e72bcbacf5a81459bd858844c319830"
server_key_iv: "291ff36b673d6001797f7683515d5f74"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.883533 22444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "485b9a233b9541eb8dbb149889b20c98"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "4e72bcbacf5a81459bd858844c319830"
server_key_iv: "291ff36b673d6001797f7683515d5f74"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.884872 22444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:44:47.885582 22459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:47.885759 22444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:47.885854 22444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "485b9a233b9541eb8dbb149889b20c98"
format_stamp: "Formatted at 2026-09-25 14:44:47 on dist-test-slave-d7lw"
server_key: "4e72bcbacf5a81459bd858844c319830"
server_key_iv: "291ff36b673d6001797f7683515d5f74"
server_key_version: "encryptionkey@0"
I20260925 14:44:47.885939 22444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:47.900516 22444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:47.902837 22444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:47.902985 22444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:47.903209 22444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:47.903491 22444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:47.903544 22444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:47.903582 22444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:47.903610 22444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:47.909112 22444 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:32895
I20260925 14:44:47.909150 22573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:32895 every 8 connection(s)
I20260925 14:44:47.909493 22444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:44:47 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347487, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:47.917659 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22444
I20260925 14:44:47.917779 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:44:47.917856 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:47.910165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:42647 (local address 127.21.205.254:33381)
0925 14:44:47.910241 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:47.910243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:47.910444 (+ 201us) server_negotiation.cc:407] Connection header received
0925 14:44:47.910833 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:47.910833 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:47.910842 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:47.910892 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:47.911697 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.912262 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.913169 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.913357 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.913416 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:47.915228 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:47.915229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:47.915229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:47.915236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:47.916663 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:47.917018 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:47.917019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:47.917019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:47.917066 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:47.917263 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:47.917263 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:47.917264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:47.917356 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:47.917403 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:47.917652 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 14:44:47.917751 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 14:44:47.917932 22327 external_mini_cluster.cc:1428] Setting key 64589690e570ab6fb1f272ae661bb21a
I20260925 14:44:47.917989 22576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:47.910153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33381 (local address 127.21.205.193:42647)
0925 14:44:47.910326 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:47.910362 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:47.910761 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:47.910944 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:47.910947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:47.911450 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:47.911620 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:47.911624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.912340 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.912341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:47.913098 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:47.913100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:47.913437 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:47.913438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:47.913547 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:47.913916 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:47.913923 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:47.915145 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:47.916743 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:47.916749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:47.916750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:47.916957 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:47.917123 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:47.917124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:47.917124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:47.917211 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:47.917409 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:47.917413 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:47.917525 (+ 112us) client_negotiation.cc:769] Sending connection context
0925 14:44:47.917599 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:44:47.917716 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260925 14:44:47.918645 22574 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33381
I20260925 14:44:47.918751 22574 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:47.918926 22574 heartbeater.cc:511] Master 127.21.205.254:33381 requested a full tablet report, sending...
I20260925 14:44:47.919400 22379 ts_manager.cc:194] Registered new tserver with Master: 485b9a233b9541eb8dbb149889b20c98 (127.21.205.193:32895)
I20260925 14:44:47.920213 22379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:42647
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:44:47.942404 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:48.019595 22579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:48.019771 22579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:48.019809 22579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:48.020859 22579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:48.020896 22579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:48.020920 22579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:44:48.022342 22579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:48.022588 22579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:48.022797 22579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:48.027616 22584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:48.027640 22585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:48.027655 22579 server_base.cc:1034] running on GCE node
W20260925 14:44:48.027637 22587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:48.028031 22579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:48.028265 22579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:48.029440 22579 hybrid_clock.cc:648] HybridClock initialized: now 1790347488029421 us; error 34 us; skew 500 ppm
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:48.031764 22579 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:44:48.032210 22579 webserver.cc:460] Webserver started at http://127.21.205.194:39481/ using document root <none> and password file <none>
I20260925 14:44:48.032437 22579 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:48.032505 22579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:48.032652 22579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:48.033638 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "4f284e9d1f6b41d18dc528d241796c7f"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "47ac6527cee7ad91bd26609253494eb7"
server_key_iv: "882b7b66b151b287e8ac4d09d63ab4ac"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.033989 22579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "4f284e9d1f6b41d18dc528d241796c7f"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "47ac6527cee7ad91bd26609253494eb7"
server_key_iv: "882b7b66b151b287e8ac4d09d63ab4ac"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.035279 22579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:44:48.036227 22594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.036370 22579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:48.036464 22579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "4f284e9d1f6b41d18dc528d241796c7f"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "47ac6527cee7ad91bd26609253494eb7"
server_key_iv: "882b7b66b151b287e8ac4d09d63ab4ac"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.036537 22579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:48.053884 22579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:48.055539 22579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:48.055688 22579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:48.055922 22579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:48.056217 22579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:48.056272 22579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.056308 22579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:48.056336 22579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.062131 22579 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:37509
I20260925 14:44:48.062177 22709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:37509 every 8 connection(s)
I20260925 14:44:48.062480 22579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:44:48.066648 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22579
I20260925 14:44:48.066758 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:44:48.066910 22327 external_mini_cluster.cc:1428] Setting key 6d864f0de4cd87bb970c4ab87963649d
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:48.071709 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.063117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59081 (local address 127.21.205.254:33381)
0925 14:44:48.063179 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:48.063181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:48.063346 (+ 165us) server_negotiation.cc:407] Connection header received
0925 14:44:48.063905 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:48.063906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:48.063917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:48.063976 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:48.064779 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.065240 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.066134 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.066310 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.066358 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.068641 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:48.068643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:48.068643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:48.068651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:48.070294 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.070709 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.070710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.070711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.070765 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.071029 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.071030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.071030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.071167 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:48.071253 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:48.071518 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 14:44:48.071605 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:44:48.071926 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.063070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33381 (local address 127.21.205.194:59081)
0925 14:44:48.063225 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:48.063262 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:48.063753 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:48.064028 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:48.064031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:48.064532 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:48.064707 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:48.064711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.065315 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.065316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:48.066066 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:48.066068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.066378 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.066379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:48.066478 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.067022 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:48.067037 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:48.068547 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:48.070392 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:48.070398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:48.070400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:48.070621 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:48.070845 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:48.070846 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:48.070846 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:48.070953 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:48.071245 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:48.071251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:48.071385 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:44:48.071493 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 14:44:48.071665 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260925 14:44:48.072379 22710 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33381
I20260925 14:44:48.072487 22710 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:48.072693 22710 heartbeater.cc:511] Master 127.21.205.254:33381 requested a full tablet report, sending...
I20260925 14:44:48.073051 22379 ts_manager.cc:194] Registered new tserver with Master: 4f284e9d1f6b41d18dc528d241796c7f (127.21.205.194:37509)
I20260925 14:44:48.073423 22379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:59081
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:44:48.090710 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:48.167748 22715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:48.167964 22715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:48.167985 22715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:48.169020 22715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:48.169057 22715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:48.169107 22715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:44:48.170511 22715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33381
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:48.170758 22715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:48.170976 22715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:44:48.175727 22715 server_base.cc:1034] running on GCE node
W20260925 14:44:48.175690 22720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:48.175697 22721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:48.175701 22723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:48.176059 22715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:48.176277 22715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:48.177429 22715 hybrid_clock.cc:648] HybridClock initialized: now 1790347488177414 us; error 22 us; skew 500 ppm
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:48.179747 22715 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:44:48.180130 22715 webserver.cc:460] Webserver started at http://127.21.205.195:44211/ using document root <none> and password file <none>
I20260925 14:44:48.180356 22715 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:48.180421 22715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:48.180569 22715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:48.181536 22715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "95dffad291544ff987c50b7abd250bc5"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "1fcca2ff8963dca0d59be9df2ead0647"
server_key_iv: "63574aaa88d96b73b36070bfcd5f014f"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.181869 22715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "95dffad291544ff987c50b7abd250bc5"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "1fcca2ff8963dca0d59be9df2ead0647"
server_key_iv: "63574aaa88d96b73b36070bfcd5f014f"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.183089 22715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:44:48.183751 22730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.183976 22715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:48.184069 22715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "95dffad291544ff987c50b7abd250bc5"
format_stamp: "Formatted at 2026-09-25 14:44:48 on dist-test-slave-d7lw"
server_key: "1fcca2ff8963dca0d59be9df2ead0647"
server_key_iv: "63574aaa88d96b73b36070bfcd5f014f"
server_key_version: "encryptionkey@0"
I20260925 14:44:48.184149 22715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:48.203233 22715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:48.205143 22715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:48.205320 22715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:48.205519 22715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:48.205801 22715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:48.205856 22715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.205892 22715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:48.205919 22715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:48.211526 22715 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:41739
I20260925 14:44:48.211611 22845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:41739 every 8 connection(s)
I20260925 14:44:48.211939 22715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:44:48.214617 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22715
I20260925 14:44:48.214696 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:44:48.214851 22327 external_mini_cluster.cc:1428] Setting key 35e688d5a349f68affb1c3f504872c6d
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:48.222277 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.212659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:51533 (local address 127.21.205.254:33381)
0925 14:44:48.212714 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:48.212716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:48.212820 (+ 104us) server_negotiation.cc:407] Connection header received
0925 14:44:48.213256 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:48.213257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:48.213266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:48.213331 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:48.214147 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.214790 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.216093 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.216289 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.216363 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.218727 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:48.218728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:48.218729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:48.218754 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:48.220611 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.221107 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.221109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.221109 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.221175 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.221524 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.221525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.221525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.221697 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:48.221768 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:48.222083 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 14:44:48.222169 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:44:48.222453 22848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.212542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33381 (local address 127.21.205.195:51533)
0925 14:44:48.212701 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:48.212737 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:48.213159 (+ 422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:48.213380 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:48.213382 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:48.213903 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:48.214072 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:48.214077 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.214875 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.214876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:48.216006 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:48.216008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.216377 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.216378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:48.216485 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.217040 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:48.217049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:48.218657 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:48.220709 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:48.220715 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:48.220717 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:48.221030 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:48.221281 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:48.221282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:48.221282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:48.221445 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:48.221767 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:48.221773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:48.221917 (+ 144us) client_negotiation.cc:769] Sending connection context
0925 14:44:48.222045 (+ 128us) client_negotiation.cc:240] Negotiation successful
0925 14:44:48.222214 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260925 14:44:48.223546 22846 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33381
I20260925 14:44:48.223629 22846 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:48.223848 22846 heartbeater.cc:511] Master 127.21.205.254:33381 requested a full tablet report, sending...
I20260925 14:44:48.224185 22379 ts_manager.cc:194] Registered new tserver with Master: 95dffad291544ff987c50b7abd250bc5 (127.21.205.195:41739)
I20260925 14:44:48.224625 22379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:51533
I20260925 14:44:48.226497 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:48.247965 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.240782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47782 (local address 127.21.205.193:32895)
0925 14:44:48.240953 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:48.240956 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:48.240965 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:48.241001 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:48.241003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:48.241015 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:48.241079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:48.241609 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.242306 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.242979 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.243263 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.243314 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.245434 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:48.245443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:48.245444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:48.245451 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:48.246934 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.247235 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.247237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.247238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.247309 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.247457 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.247457 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.247457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.247573 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:48.247692 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:48.247735 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 14:44:48.247884 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
W20260925 14:44:48.248329 22487 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47782
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22335](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:48.268631 22853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:48.262240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.21.205.193:32895)
0925 14:44:48.262298 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:48.262300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:48.262329 (+ 29us) server_negotiation.cc:407] Connection header received
0925 14:44:48.262393 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:48.262394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:48.262402 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:48.262447 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:48.262945 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.263381 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.264185 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:48.264374 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:48.264423 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:48.266449 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:48.266450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:48.266451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:48.266457 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:48.267752 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.268102 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.268103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.268103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.268148 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:48.268283 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:48.268284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:48.268284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:48.268362 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:48.268436 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:48.268499 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 14:44:48.268552 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:44:48.269173 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22444
I20260925 14:44:48.274511 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22579
I20260925 14:44:48.280759 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22715
I20260925 14:44:48.285066 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22348
2026-09-25T14:44:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1857 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22860](info): set up 2 sockets
Sep 25 14:44:48 dist-test-slave-d7lw krb5kdc[22860](info): commencing operation
krb5kdc: starting...
W20260925 14:44:49.302821 22327 mini_kdc.cc:121] Time spent starting KDC: real 0.998s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:44:49.410454 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33515
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33515
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:49.488807 22873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:49.488982 22873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:49.489003 22873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:49.490090 22873 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:44:49.490146 22873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:49.490160 22873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:49.490172 22873 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:44:49.490183 22873 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:44:49.491626 22873 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:49.491891 22873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:49.492091 22873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:49.497025 22879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.496999 22881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.497040 22873 server_base.cc:1034] running on GCE node
W20260925 14:44:49.497010 22878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.497427 22873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:49.497740 22873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:49.498905 22873 hybrid_clock.cc:648] HybridClock initialized: now 1790347489498900 us; error 35 us; skew 500 ppm
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:49.501291 22873 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:44:49.501699 22873 webserver.cc:460] Webserver started at http://127.21.205.254:36287/ using document root <none> and password file <none>
I20260925 14:44:49.501863 22873 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:49.501899 22873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:49.501978 22873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:49.502892 22873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "c8835804acd044569875c843270abc79"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "18a0a69a589bac423f1a8b76575098a9"
server_key_iv: "b1ac29b195169ed63679638a8c9387c5"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.503176 22873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "c8835804acd044569875c843270abc79"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "18a0a69a589bac423f1a8b76575098a9"
server_key_iv: "b1ac29b195169ed63679638a8c9387c5"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.504549 22873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:44:49.505206 22888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.505391 22873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:49.505460 22873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "c8835804acd044569875c843270abc79"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "18a0a69a589bac423f1a8b76575098a9"
server_key_iv: "b1ac29b195169ed63679638a8c9387c5"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.505507 22873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:49.519574 22873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:49.521713 22873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:49.521813 22873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:49.525410 22873 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33515
I20260925 14:44:49.525503 22941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33515 every 8 connection(s)
I20260925 14:44:49.525800 22873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:44:49.526556 22942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:49.528741 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79: Bootstrap starting.
I20260925 14:44:49.529223 22942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:49.529446 22942 log.cc:824] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:49.530126 22942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79: No bootstrap required, opened a new log
I20260925 14:44:49.531703 22942 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } }
I20260925 14:44:49.531904 22942 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:49.531935 22942 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8835804acd044569875c843270abc79, State: Initialized, Role: FOLLOWER
I20260925 14:44:49.532039 22942 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } }
I20260925 14:44:49.532094 22942 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:44:49.532115 22942 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:44:49.532153 22942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:49.532621 22942 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } }
I20260925 14:44:49.532728 22942 leader_election.cc:302] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8835804acd044569875c843270abc79; no voters:
I20260925 14:44:49.532845 22942 leader_election.cc:288] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:44:49.532904 22946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:49.533054 22946 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [term 1 LEADER]: Becoming Leader. State: Replica: c8835804acd044569875c843270abc79, State: Running, Role: LEADER
I20260925 14:44:49.533113 22942 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:44:49.533131 22946 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } }
I20260925 14:44:49.533531 22948 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8835804acd044569875c843270abc79. Latest consensus state: current_term: 1 leader_uuid: "c8835804acd044569875c843270abc79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } } }
I20260925 14:44:49.533612 22948 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:49.533772 22947 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8835804acd044569875c843270abc79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8835804acd044569875c843270abc79" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33515 } } }
I20260925 14:44:49.533921 22947 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79 [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:49.533995 22952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:44:49.534447 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22873
I20260925 14:44:49.534487 22952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:44:49.534550 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:44:49.534739 22327 external_mini_cluster.cc:1428] Setting key 328a8cb072b186681530a15c7d7ab283
I20260925 14:44:49.536249 22952 catalog_manager.cc:1347] Generated new cluster ID: 317272f8635e4968b30007c0eb6cb038
I20260925 14:44:49.536329 22952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:49.543467 22952 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:44:49.544075 22952 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:44:49.546411 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:49.535198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51364 (local address 127.21.205.254:33515)
0925 14:44:49.535369 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:49.535377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:49.535408 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:44:49.535527 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:49.535533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:49.535592 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:49.535977 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:49.536682 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.538093 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.539194 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.539766 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.539902 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:49.542659 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:49.542663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:49.542663 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:49.542675 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:49.544779 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.545178 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.545180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.545181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.545284 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.545498 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.545499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.545499 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.545773 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:49.545921 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:49.546041 (+ 120us) server_negotiation.cc:299] Negotiation successful
0925 14:44:49.546195 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260925 14:44:49.550758 22952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8835804acd044569875c843270abc79: Generated new TSK 0
I20260925 14:44:49.550932 22952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:44:49.576323 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:49.653558 22969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:49.653738 22969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:49.653762 22969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:49.654910 22969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:49.654945 22969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:49.654979 22969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:44:49.656461 22969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:49.656685 22969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:49.656931 22969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:49.662425 22975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.662421 22974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.662478 22977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.662750 22969 server_base.cc:1034] running on GCE node
I20260925 14:44:49.662950 22969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:49.663205 22969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:49.664397 22969 hybrid_clock.cc:648] HybridClock initialized: now 1790347489664385 us; error 24 us; skew 500 ppm
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:49.666610 22969 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:44:49.666955 22969 webserver.cc:460] Webserver started at http://127.21.205.193:42217/ using document root <none> and password file <none>
I20260925 14:44:49.667125 22969 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:49.667162 22969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:49.667251 22969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:49.668221 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "7d964276f13047ffb27c94780a1b5364"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "eed98d911c0e04fdc26dc1a1ae429b9e"
server_key_iv: "6171e5a976674e0f37ddfbf57596f52e"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.668483 22969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "7d964276f13047ffb27c94780a1b5364"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "eed98d911c0e04fdc26dc1a1ae429b9e"
server_key_iv: "6171e5a976674e0f37ddfbf57596f52e"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.669631 22969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:49.670264 22985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.670444 22969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:49.670526 22969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "7d964276f13047ffb27c94780a1b5364"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "eed98d911c0e04fdc26dc1a1ae429b9e"
server_key_iv: "6171e5a976674e0f37ddfbf57596f52e"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.670578 22969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:49.695739 22969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:49.698076 22969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:49.698199 22969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:49.698372 22969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:49.698626 22969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:49.698664 22969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.698688 22969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:49.698700 22969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.704782 22969 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:44421
I20260925 14:44:49.704865 23100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:44421 every 8 connection(s)
I20260925 14:44:49.705189 22969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:44:49.710330 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 22969
I20260925 14:44:49.710422 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:44:49.710575 22327 external_mini_cluster.cc:1428] Setting key c4f3a7bb36242ed7e847eb8b8468b1b4
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:49.715054 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:49.705844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:60667 (local address 127.21.205.254:33515)
0925 14:44:49.705905 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:49.705907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:49.706107 (+ 200us) server_negotiation.cc:407] Connection header received
0925 14:44:49.707048 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:49.707050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:49.707064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:49.707117 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:49.707739 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.708419 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.709602 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.709829 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.709919 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:49.712146 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:49.712148 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:49.712148 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:49.712156 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:49.713617 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.714058 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.714059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.714060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.714123 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.714387 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.714388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.714388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.714505 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:49.714580 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:49.714870 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 14:44:49.714963 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:44:49.715211 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:49.705840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33515 (local address 127.21.205.193:60667)
0925 14:44:49.705989 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:49.706022 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:49.706930 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:49.707207 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:49.707211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:49.707457 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:49.707632 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:49.707637 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.708542 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.708543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:49.709492 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:49.709495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.709932 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.709933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:49.710057 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:49.710563 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:49.710571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:49.712032 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:49.713729 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:49.713737 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:49.713739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:49.713963 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:49.714197 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:49.714198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:49.714199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:49.714313 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:49.714586 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:49.714593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:49.714727 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:44:49.714830 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 14:44:49.714980 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260925 14:44:49.715687 23101 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33515
I20260925 14:44:49.715806 23101 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:49.716017 23101 heartbeater.cc:511] Master 127.21.205.254:33515 requested a full tablet report, sending...
I20260925 14:44:49.716621 22905 ts_manager.cc:194] Registered new tserver with Master: 7d964276f13047ffb27c94780a1b5364 (127.21.205.193:44421)
I20260925 14:44:49.717231 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:60667
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:44:49.732564 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:49.816710 23106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:49.816895 23106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:49.816917 23106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:49.818043 23106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:49.818082 23106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:49.818109 23106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:44:49.819555 23106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:49.819811 23106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:49.820039 23106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:49.824762 23114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.824843 23112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.824842 23106 server_base.cc:1034] running on GCE node
W20260925 14:44:49.824762 23111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.825205 23106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:49.825436 23106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:49.826586 23106 hybrid_clock.cc:648] HybridClock initialized: now 1790347489826570 us; error 45 us; skew 500 ppm
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:49.828852 23106 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:44:49.829185 23106 webserver.cc:460] Webserver started at http://127.21.205.194:34993/ using document root <none> and password file <none>
I20260925 14:44:49.829360 23106 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:49.829396 23106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:49.829475 23106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:49.830366 23106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "56c35cb5c58948a99bb41fe931018624"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "d3c52377b1e09afbc4a424fc600651a5"
server_key_iv: "9fcd822c598308e588da30f85d3a7bde"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.830636 23106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "56c35cb5c58948a99bb41fe931018624"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "d3c52377b1e09afbc4a424fc600651a5"
server_key_iv: "9fcd822c598308e588da30f85d3a7bde"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.831830 23106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:49.832489 23121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.832638 23106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:49.832775 23106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "56c35cb5c58948a99bb41fe931018624"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "d3c52377b1e09afbc4a424fc600651a5"
server_key_iv: "9fcd822c598308e588da30f85d3a7bde"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.832845 23106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:49.843490 23106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:49.845141 23106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:49.845242 23106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:49.845386 23106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:49.845650 23106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:49.845682 23106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.845702 23106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:49.845748 23106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.851457 23106 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:35951
I20260925 14:44:49.851531 23236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:35951 every 8 connection(s)
I20260925 14:44:49.851907 23106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:44:49.856653 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23106
I20260925 14:44:49.856755 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:44:49.856931 22327 external_mini_cluster.cc:1428] Setting key f9ef095d9bcab0d1ee8e0ed64a2c7b8f
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:49.861665 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:49.852668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:49115 (local address 127.21.205.254:33515)
0925 14:44:49.852724 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:49.852726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:49.852876 (+ 150us) server_negotiation.cc:407] Connection header received
0925 14:44:49.853717 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:49.853718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:49.853727 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:49.853779 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:49.854313 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.854803 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.855703 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.855945 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.855994 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:49.858369 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:49.858370 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:49.858370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:49.858377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:49.859943 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.860442 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.860443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.860443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.860491 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:49.860781 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:49.860782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:49.860783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:49.860962 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:49.861038 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:49.861367 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 14:44:49.861507 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:44:49.861950 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:49.852614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33515 (local address 127.21.205.194:49115)
0925 14:44:49.852771 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:49.852808 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:49.853628 (+ 820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:49.853826 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:49.853829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:49.854066 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:49.854242 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:49.854246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.854876 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.854877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:49.855630 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:49.855631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:49.856011 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:49.856012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:49.856127 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:49.856527 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:49.856534 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:49.858267 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:49.860044 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:49.860050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:49.860051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:49.860358 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:49.860555 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:49.860556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:49.860556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:49.860689 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:49.861027 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:49.861032 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:49.861202 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 14:44:49.861315 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 14:44:49.861680 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":83,"threads_started":1}
I20260925 14:44:49.862411 23237 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33515
I20260925 14:44:49.862507 23237 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:49.862689 23237 heartbeater.cc:511] Master 127.21.205.254:33515 requested a full tablet report, sending...
I20260925 14:44:49.863080 22905 ts_manager.cc:194] Registered new tserver with Master: 56c35cb5c58948a99bb41fe931018624 (127.21.205.194:35951)
I20260925 14:44:49.863466 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:49115
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:44:49.882344 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:49.962352 23242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:49.962533 23242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:49.962556 23242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:49.963717 23242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:49.963758 23242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:49.963809 23242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:44:49.965315 23242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33515
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:49.965550 23242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:49.965792 23242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:49.971724 23249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.971724 23248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:49.971838 23251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:49.971941 23242 server_base.cc:1034] running on GCE node
I20260925 14:44:49.972131 23242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:49.972347 23242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:49.973510 23242 hybrid_clock.cc:648] HybridClock initialized: now 1790347489973498 us; error 28 us; skew 500 ppm
Sep 25 14:44:49 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:49.975871 23242 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:44:49.976306 23242 webserver.cc:460] Webserver started at http://127.21.205.195:34631/ using document root <none> and password file <none>
I20260925 14:44:49.976478 23242 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:49.976516 23242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:49.976596 23242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:49.977578 23242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "a62bd5c26b4f471ab621bd31f4a64d9d"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "588277279df6137edef07303053cc125"
server_key_iv: "343793fb9aaf56daf92cdb90f31ade5d"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.977912 23242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "a62bd5c26b4f471ab621bd31f4a64d9d"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "588277279df6137edef07303053cc125"
server_key_iv: "343793fb9aaf56daf92cdb90f31ade5d"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.979151 23242 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:44:49.979828 23259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.980005 23242 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:49.980113 23242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "a62bd5c26b4f471ab621bd31f4a64d9d"
format_stamp: "Formatted at 2026-09-25 14:44:49 on dist-test-slave-d7lw"
server_key: "588277279df6137edef07303053cc125"
server_key_iv: "343793fb9aaf56daf92cdb90f31ade5d"
server_key_version: "encryptionkey@0"
I20260925 14:44:49.980206 23242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:49.993399 23242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:49.995043 23242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:49.995190 23242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:49.995388 23242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:44:49.995659 23242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:49.995718 23242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:49.995755 23242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:49.995806 23242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:50.001763 23242 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:45915
I20260925 14:44:50.001809 23373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:45915 every 8 connection(s)
I20260925 14:44:50.002138 23242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:44:50.006484 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23242
I20260925 14:44:50.006567 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:44:50.006723 22327 external_mini_cluster.cc:1428] Setting key 72a85d0db7dc3954f4da59292f16eb0f
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:50.011123 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.002775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:43623 (local address 127.21.205.254:33515)
0925 14:44:50.002836 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.002839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.003032 (+ 193us) server_negotiation.cc:407] Connection header received
0925 14:44:50.003935 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.003935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.003946 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.003998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:50.004518 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.005023 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.006063 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.006252 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.006320 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.008756 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:50.008757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:50.008757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:50.008762 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:50.010015 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.010411 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.010411 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.010412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.010457 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.010651 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.010652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.010652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.010752 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:50.010798 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.011008 (+ 210us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.011063 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:44:50.011307 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.002751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33515 (local address 127.21.205.195:43623)
0925 14:44:50.002918 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.002953 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.003843 (+ 890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.004047 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.004050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.004272 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:50.004450 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.004453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.005095 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.005096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.005985 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.005987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.006329 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.006330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.006438 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.007284 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:50.007298 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:50.008669 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:50.010095 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:50.010100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:50.010101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:50.010350 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:50.010515 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:50.010516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:50.010516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:50.010601 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:50.010804 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:50.010808 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:50.010906 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.010981 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.011084 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260925 14:44:50.011757 23374 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33515
I20260925 14:44:50.011885 23374 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:50.012055 23374 heartbeater.cc:511] Master 127.21.205.254:33515 requested a full tablet report, sending...
I20260925 14:44:50.012399 22905 ts_manager.cc:194] Registered new tserver with Master: a62bd5c26b4f471ab621bd31f4a64d9d (127.21.205.195:45915)
I20260925 14:44:50.012759 22905 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:43623
I20260925 14:44:50.018409 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:44:50.026569 22965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.020164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.21.205.254:33515)
0925 14:44:50.020237 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.020239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.020298 (+ 59us) server_negotiation.cc:407] Connection header received
0925 14:44:50.020346 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.020346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.020354 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.020399 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:50.020943 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.021391 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.022083 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.022254 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.022307 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.023552 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:50.023554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:50.023554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:50.023560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:50.025445 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.025781 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.025782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.025782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.025836 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.026007 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.026007 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.026008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.026188 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:50.026283 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.026363 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.026440 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 14:44:50.028905 22905 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:44:50.030498 22905 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:50.037173 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.032822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45544 (local address 127.21.205.195:45915)
0925 14:44:50.033269 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.033272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.033306 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:44:50.033339 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.033340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.033354 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.033466 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:50.033657 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.034846 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036030 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.036779 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036864 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.036915 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.037053 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.037082 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":80,"threads_started":1}
I20260925 14:44:50.037175 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.032665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:45915 (local address 127.0.0.1:45544)
0925 14:44:50.032915 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.032926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.032980 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.033460 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.033460 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.033465 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:50.033554 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.033557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.035045 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.035046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.035923 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.035926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.036867 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.036951 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.036967 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.037041 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.037091 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":19,"thread_start_us":57,"threads_started":1}
I20260925 14:44:50.037273 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.032665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:44421 (local address 127.0.0.1:57168)
0925 14:44:50.032826 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.032861 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.032922 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.033328 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.033331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.033333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:50.033411 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.033413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.034689 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.034690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.036189 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.036190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.037039 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.037040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.037093 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.037095 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.037147 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.037178 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":2,"thread_start_us":62,"threads_started":1}
I20260925 14:44:50.037614 23382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.032704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:35951 (local address 127.0.0.1:38504)
0925 14:44:50.033018 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.033026 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.033063 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.033459 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.033460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.033463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:50.033647 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.033649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.034965 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.034966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.036013 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.036015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.037386 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.037387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.037444 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.037453 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.037519 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.037556 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":111,"thread_start_us":77,"threads_started":1}
I20260925 14:44:50.037711 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.032860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.21.205.193:44421)
0925 14:44:50.033072 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.033077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.033086 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:50.033157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.033159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.033176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.033314 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:50.033520 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.034547 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036260 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.036890 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036956 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.037007 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.037186 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.037606 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260925 14:44:50.037921 23307 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0137c5ab664ce7a36ad492a17e5505 (DEFAULT_TABLE table=test-table [id=836a733b97a740f4bd242ee600f768ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:50.038066 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.033001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38504 (local address 127.21.205.194:35951)
0925 14:44:50.033188 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.033192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.033202 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:44:50.033240 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.033243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.033260 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.033334 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:50.033740 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.034842 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036078 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.036781 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.036864 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.036915 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.037937 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.037998 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260925 14:44:50.038316 23307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0137c5ab664ce7a36ad492a17e5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:50.038403 23033 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0137c5ab664ce7a36ad492a17e5505 (DEFAULT_TABLE table=test-table [id=836a733b97a740f4bd242ee600f768ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:50.038736 23033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0137c5ab664ce7a36ad492a17e5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:50.040215 23169 tablet_service.cc:1467] Processing CreateTablet for tablet 9b0137c5ab664ce7a36ad492a17e5505 (DEFAULT_TABLE table=test-table [id=836a733b97a740f4bd242ee600f768ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:50.040561 23169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b0137c5ab664ce7a36ad492a17e5505. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:50.041189 23389 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Bootstrap starting.
I20260925 14:44:50.041831 23390 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Bootstrap starting.
I20260925 14:44:50.041934 23389 tablet_bootstrap.cc:654] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:50.042492 23390 tablet_bootstrap.cc:654] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:50.042608 23389 log.cc:824] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:50.043027 23390 log.cc:824] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:50.043480 23389 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: No bootstrap required, opened a new log
I20260925 14:44:50.043560 23389 ts_tablet_manager.cc:1397] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:50.043769 23390 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: No bootstrap required, opened a new log
I20260925 14:44:50.043860 23390 ts_tablet_manager.cc:1397] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:50.044255 23393 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Bootstrap starting.
I20260925 14:44:50.044857 23393 tablet_bootstrap.cc:654] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:50.045298 23389 raft_consensus.cc:348] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.045425 23393 log.cc:824] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:50.045456 23389 raft_consensus.cc:374] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:50.045501 23389 raft_consensus.cc:729] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a62bd5c26b4f471ab621bd31f4a64d9d, State: Initialized, Role: FOLLOWER
I20260925 14:44:50.045606 23389 consensus_queue.cc:260] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.045892 23389 ts_tablet_manager.cc:1428] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 14:44:50.046016 23374 heartbeater.cc:503] Master 127.21.205.254:33515 was elected leader, sending a full tablet report...
I20260925 14:44:50.046028 23390 raft_consensus.cc:348] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.046204 23393 tablet_bootstrap.cc:492] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: No bootstrap required, opened a new log
I20260925 14:44:50.046205 23390 raft_consensus.cc:374] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:50.046242 23390 raft_consensus.cc:729] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d964276f13047ffb27c94780a1b5364, State: Initialized, Role: FOLLOWER
I20260925 14:44:50.046346 23390 consensus_queue.cc:260] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.046267 23393 ts_tablet_manager.cc:1397] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:50.046615 23390 ts_tablet_manager.cc:1428] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:44:50.046646 23101 heartbeater.cc:503] Master 127.21.205.254:33515 was elected leader, sending a full tablet report...
I20260925 14:44:50.048679 23393 raft_consensus.cc:348] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.048818 23393 raft_consensus.cc:374] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:50.048839 23393 raft_consensus.cc:729] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c35cb5c58948a99bb41fe931018624, State: Initialized, Role: FOLLOWER
I20260925 14:44:50.048905 23393 consensus_queue.cc:260] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.049077 23393 ts_tablet_manager.cc:1428] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:44:50.049129 23237 heartbeater.cc:503] Master 127.21.205.254:33515 was elected leader, sending a full tablet report...
I20260925 14:44:50.051723 23396 raft_consensus.cc:482] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:50.051860 23396 raft_consensus.cc:504] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.052201 23396 leader_election.cc:288] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a62bd5c26b4f471ab621bd31f4a64d9d (127.21.205.195:45915), 56c35cb5c58948a99bb41fe931018624 (127.21.205.194:35951)
I20260925 14:44:50.055138 23103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.052294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:45915 (local address 127.21.205.193:35567)
0925 14:44:50.052447 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.052457 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.052504 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.052641 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.052642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.052644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:50.052725 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.052727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.053447 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.053449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.054299 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.054302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.054868 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.054869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.054945 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.054949 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.055012 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.055073 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260925 14:44:50.055147 23386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.052380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:35567 (local address 127.21.205.195:45915)
0925 14:44:50.052440 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.052442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.052489 (+ 47us) server_negotiation.cc:407] Connection header received
0925 14:44:50.052569 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.052570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.052579 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.052625 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:50.052818 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.053303 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.054375 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.054794 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.054843 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.054848 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.055036 (+ 188us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.055085 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:44:50.055317 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.052587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:34173 (local address 127.21.205.194:35951)
0925 14:44:50.052668 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.052670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.052679 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:50.052711 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.052712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.052721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.052775 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:50.052964 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.053484 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.054534 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.054959 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.055020 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.055027 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.055205 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.055247 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 14:44:50.055317 23398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.052333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:35951 (local address 127.21.205.193:34173)
0925 14:44:50.052465 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:50.052483 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:50.052526 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:50.052813 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:50.052814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:50.052815 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:50.052878 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.052879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.053565 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.053566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.054454 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:50.054456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.055041 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.055042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:50.055129 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.055134 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:44:50.055189 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:44:50.055230 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260925 14:44:50.055437 23327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0137c5ab664ce7a36ad492a17e5505" candidate_uuid: "7d964276f13047ffb27c94780a1b5364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" is_pre_election: true
I20260925 14:44:50.055575 23327 raft_consensus.cc:2454] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d964276f13047ffb27c94780a1b5364 in term 0.
I20260925 14:44:50.055680 23189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0137c5ab664ce7a36ad492a17e5505" candidate_uuid: "7d964276f13047ffb27c94780a1b5364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56c35cb5c58948a99bb41fe931018624" is_pre_election: true
I20260925 14:44:50.055765 22989 leader_election.cc:302] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d964276f13047ffb27c94780a1b5364, a62bd5c26b4f471ab621bd31f4a64d9d; no voters:
I20260925 14:44:50.055819 23189 raft_consensus.cc:2454] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d964276f13047ffb27c94780a1b5364 in term 0.
I20260925 14:44:50.055905 23396 raft_consensus.cc:2781] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:50.055968 23396 raft_consensus.cc:482] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:50.056005 23396 raft_consensus.cc:3037] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 0 FOLLOWER]: Advancing to term 1
W20260925 14:44:50.102487 23238 tablet.cc:2369] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:44:50.125851 23396 raft_consensus.cc:504] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.126068 23396 leader_election.cc:288] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [CANDIDATE]: Term 1 election: Requested vote from peers a62bd5c26b4f471ab621bd31f4a64d9d (127.21.205.195:45915), 56c35cb5c58948a99bb41fe931018624 (127.21.205.194:35951)
I20260925 14:44:50.126240 23327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0137c5ab664ce7a36ad492a17e5505" candidate_uuid: "7d964276f13047ffb27c94780a1b5364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d"
I20260925 14:44:50.126313 23189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b0137c5ab664ce7a36ad492a17e5505" candidate_uuid: "7d964276f13047ffb27c94780a1b5364" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56c35cb5c58948a99bb41fe931018624"
I20260925 14:44:50.126364 23327 raft_consensus.cc:3037] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:50.126439 23189 raft_consensus.cc:3037] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:50.127180 23327 raft_consensus.cc:2454] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d964276f13047ffb27c94780a1b5364 in term 1.
I20260925 14:44:50.127393 22989 leader_election.cc:302] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d964276f13047ffb27c94780a1b5364, a62bd5c26b4f471ab621bd31f4a64d9d; no voters:
I20260925 14:44:50.127506 23396 raft_consensus.cc:2781] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:50.127517 23189 raft_consensus.cc:2454] T 9b0137c5ab664ce7a36ad492a17e5505 P 56c35cb5c58948a99bb41fe931018624 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d964276f13047ffb27c94780a1b5364 in term 1.
I20260925 14:44:50.127678 23396 raft_consensus.cc:686] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [term 1 LEADER]: Becoming Leader. State: Replica: 7d964276f13047ffb27c94780a1b5364, State: Running, Role: LEADER
I20260925 14:44:50.127830 23396 consensus_queue.cc:237] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } }
I20260925 14:44:50.128769 22904 catalog_manager.cc:5495] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d964276f13047ffb27c94780a1b5364 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "7d964276f13047ffb27c94780a1b5364" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7d964276f13047ffb27c94780a1b5364" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 44421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a62bd5c26b4f471ab621bd31f4a64d9d" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56c35cb5c58948a99bb41fe931018624" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 35951 } health_report { overall_health: UNKNOWN } } }
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:50.154745 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.147762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.21.205.193:44421)
0925 14:44:50.147878 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.147880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.147890 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:44:50.147960 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.147961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.147969 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.148016 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:50.148568 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.149036 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.149746 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.149924 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.149970 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.152234 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:50.152243 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:50.152243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:50.152268 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:50.153795 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.154098 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.154101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.154101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.154154 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.154317 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.154318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.154318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.154443 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:50.154542 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.154596 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.154665 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347490, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260925 14:44:50.206008 23102 tablet.cc:2369] T 9b0137c5ab664ce7a36ad492a17e5505 P 7d964276f13047ffb27c94780a1b5364: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347490, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:50.233505 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.225772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.21.205.193:44421)
0925 14:44:50.225837 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.225839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.225914 (+ 75us) server_negotiation.cc:407] Connection header received
0925 14:44:50.225961 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.225962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.225974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.226034 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:50.226597 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.227428 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.228269 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.228666 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.228730 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.231115 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:50.231117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:50.231117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:50.231124 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:50.232364 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.232753 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.232754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.232755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.232812 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.232977 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.232978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.232978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.233072 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:50.233156 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.233364 (+ 208us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.233439 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260925 14:44:50.252641 23375 tablet.cc:2369] T 9b0137c5ab664ce7a36ad492a17e5505 P a62bd5c26b4f471ab621bd31f4a64d9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 14:44:50 dist-test-slave-d7lw krb5kdc[22860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:50.332407 23387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:50.324968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57188 (local address 127.21.205.193:44421)
0925 14:44:50.325017 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:50.325019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:50.325110 (+ 91us) server_negotiation.cc:407] Connection header received
0925 14:44:50.325166 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:50.325167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:50.325176 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:50.325227 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:50.325972 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.326455 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.327715 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:50.327954 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:50.328009 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:50.330353 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:50.330355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:50.330355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:50.330361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:50.331600 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.331885 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.331886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.331886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.331932 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:50.332071 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:50.332071 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:50.332072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:50.332147 (+ 75us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:50.332217 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:50.332291 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 14:44:50.332343 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260925 14:44:50.333092 23013 tablet_service.cc:2908] Rejecting scan request for tablet 9b0137c5ab664ce7a36ad492a17e5505: Uninitialized: clean time has not yet been initialized
I20260925 14:44:50.333964 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22969
I20260925 14:44:50.340087 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23106
I20260925 14:44:50.344625 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23242
I20260925 14:44:50.348878 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 22873
2026-09-25T14:44:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2064 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:44:51 dist-test-slave-d7lw krb5kdc[23413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:44:51 dist-test-slave-d7lw krb5kdc[23413](info): set up 2 sockets
Sep 25 14:44:51 dist-test-slave-d7lw krb5kdc[23413](info): commencing operation
krb5kdc: starting...
W20260925 14:44:52.983904 22327 mini_kdc.cc:121] Time spent starting KDC: real 2.614s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260925 14:44:53.092504 22327 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.108s user 0.001s sys 0.001s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260925 14:44:53.204698 22327 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.112s user 0.001s sys 0.001s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260925 14:44:53.322865 22327 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.118s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/test-user.keytab.
W20260925 14:44:53.681468 22327 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.358s user 0.000s sys 0.001s
Sep 25 14:44:53 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
W20260925 14:44:54.487748 22327 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.205.254: real 0.786s user 0.000s sys 0.002s
I20260925 14:44:54.488237 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:39085
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:39085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:54.574915 23426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:54.575095 23426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:54.575145 23426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:54.576396 23426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:44:54.576434 23426 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 14:44:54.576448 23426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:54.576478 23426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:54.576504 23426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:44:54.576520 23426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:44:54.578012 23426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:39085
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:39085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:54.578240 23426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:54.578464 23426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:54.583253 23432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:54.583292 23434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:54.583244 23431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:54.583320 23426 server_base.cc:1034] running on GCE node
I20260925 14:44:54.583633 23426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:54.583956 23426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:54.585116 23426 hybrid_clock.cc:648] HybridClock initialized: now 1790347494585102 us; error 47 us; skew 500 ppm
Sep 25 14:44:54 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:54.587497 23426 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:44:54.587983 23426 webserver.cc:460] Webserver started at http://127.21.205.254:43623/ using document root <none> and password file <none>
I20260925 14:44:54.588231 23426 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:54.588299 23426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:54.588426 23426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:54.589460 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "7c4eed5a3ca44f9383456694a04baec2"
format_stamp: "Formatted at 2026-09-25 14:44:54 on dist-test-slave-d7lw"
server_key: "ee2e6aec06ba8e9e42613e785dcba1c4"
server_key_iv: "3635d0bce5861ced971d021e2910e678"
server_key_version: "encryptionkey@0"
I20260925 14:44:54.589828 23426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "7c4eed5a3ca44f9383456694a04baec2"
format_stamp: "Formatted at 2026-09-25 14:44:54 on dist-test-slave-d7lw"
server_key: "ee2e6aec06ba8e9e42613e785dcba1c4"
server_key_iv: "3635d0bce5861ced971d021e2910e678"
server_key_version: "encryptionkey@0"
I20260925 14:44:54.591159 23426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:44:54.591997 23441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:54.592171 23426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:54.592275 23426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "7c4eed5a3ca44f9383456694a04baec2"
format_stamp: "Formatted at 2026-09-25 14:44:54 on dist-test-slave-d7lw"
server_key: "ee2e6aec06ba8e9e42613e785dcba1c4"
server_key_iv: "3635d0bce5861ced971d021e2910e678"
server_key_version: "encryptionkey@0"
I20260925 14:44:54.592355 23426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:54.608505 23426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:54.610177 23426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:54.610327 23426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:54.614723 23426 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:39085
I20260925 14:44:54.614842 23504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:39085 every 8 connection(s)
I20260925 14:44:54.615088 23426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:44:54.615808 23505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:54.618034 23505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Bootstrap starting.
I20260925 14:44:54.618680 23505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:54.618983 23505 log.cc:824] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:54.619628 23505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: No bootstrap required, opened a new log
I20260925 14:44:54.621675 23505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } }
I20260925 14:44:54.621807 23505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:54.621886 23505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4eed5a3ca44f9383456694a04baec2, State: Initialized, Role: FOLLOWER
I20260925 14:44:54.622036 23505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } }
I20260925 14:44:54.622097 23505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:44:54.622138 23505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:44:54.622185 23505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:54.623106 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23426
I20260925 14:44:54.623181 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:44:54.623346 22327 external_mini_cluster.cc:1428] Setting key c40440c62c90a4b4684b145277e18bee
Sep 25 14:44:54 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:54.631322 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:54.623800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.21.205.254:39085)
0925 14:44:54.623963 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:54.623971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:54.624003 (+ 32us) server_negotiation.cc:407] Connection header received
0925 14:44:54.624088 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:54.624093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:54.624132 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:54.624233 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:54.624752 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:54.625560 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:54.626244 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:54.626601 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:54.626672 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:54.628682 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:54.628685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:54.628685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:54.628694 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:54.630020 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:54.630323 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:54.630325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:54.630325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:54.630398 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:54.630569 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:54.630569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:54.630570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:54.630811 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:54.630902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:54.630997 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 14:44:54.631095 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260925 14:44:54.688648 23505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } }
I20260925 14:44:54.688843 23505 leader_election.cc:302] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c4eed5a3ca44f9383456694a04baec2; no voters:
I20260925 14:44:54.689095 23505 leader_election.cc:288] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:44:54.689182 23510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:54.689404 23505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:44:54.689401 23510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [term 1 LEADER]: Becoming Leader. State: Replica: 7c4eed5a3ca44f9383456694a04baec2, State: Running, Role: LEADER
I20260925 14:44:54.689559 23510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } }
I20260925 14:44:54.690063 23511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c4eed5a3ca44f9383456694a04baec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } } }
I20260925 14:44:54.690162 23511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:54.690794 23519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:44:54.691030 23512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c4eed5a3ca44f9383456694a04baec2. Latest consensus state: current_term: 1 leader_uuid: "7c4eed5a3ca44f9383456694a04baec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c4eed5a3ca44f9383456694a04baec2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39085 } } }
I20260925 14:44:54.691198 23512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:54.691309 23519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:44:54.693137 23519 catalog_manager.cc:1347] Generated new cluster ID: 5fd0196589c34117b0db55a0e4d6a0a9
I20260925 14:44:54.693192 23519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:44:54.712261 23519 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:44:54.712999 23519 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:44:54.719396 23519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Generated new TSK 0
I20260925 14:44:54.719563 23519 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
W20260925 14:44:55.422032 22327 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.205.193: real 0.697s user 0.000s sys 0.003s
I20260925 14:44:55.422387 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:39085
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:55.500332 23531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:55.500499 23531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:55.500520 23531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:55.501557 23531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:55.501588 23531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:55.501607 23531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:55.501623 23531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:44:55.502969 23531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:55.503132 23531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:55.503295 23531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:55.507982 23540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:55.507973 23537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:55.507977 23538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:55.507989 23531 server_base.cc:1034] running on GCE node
I20260925 14:44:55.508322 23531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:55.508548 23531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:55.509717 23531 hybrid_clock.cc:648] HybridClock initialized: now 1790347495509699 us; error 31 us; skew 500 ppm
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:55.512406 23531 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:44:55.512816 23531 webserver.cc:460] Webserver started at http://127.21.205.193:33721/ using document root <none> and password file <none>
I20260925 14:44:55.513055 23531 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:55.513134 23531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:55.513293 23531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:55.514307 23531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "43a40bbe2c8f4a1b924e990cc836be96"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "08aea4d7f7c80e413a691ccc2609eb31"
server_key_iv: "6d6b2ded0a0dc5b0f413adaa016a18ce"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.514680 23531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "43a40bbe2c8f4a1b924e990cc836be96"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "08aea4d7f7c80e413a691ccc2609eb31"
server_key_iv: "6d6b2ded0a0dc5b0f413adaa016a18ce"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.516075 23531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:44:55.516875 23547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.517048 23531 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:55.517145 23531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "43a40bbe2c8f4a1b924e990cc836be96"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "08aea4d7f7c80e413a691ccc2609eb31"
server_key_iv: "6d6b2ded0a0dc5b0f413adaa016a18ce"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.517230 23531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:55.537595 23531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:55.540009 23531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:55.540186 23531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:55.540688 23531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:55.540774 23531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.540824 23531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:55.540853 23531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:55.548507 23531 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:33945
I20260925 14:44:55.548806 23669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:33945 every 8 connection(s)
I20260925 14:44:55.548866 23531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:44:55.551030 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.541221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:37913 (local address 127.21.205.254:39085)
0925 14:44:55.541346 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.541348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.541529 (+ 181us) server_negotiation.cc:407] Connection header received
0925 14:44:55.542040 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.542041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.542055 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.542134 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.543062 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.543935 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.545516 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.545837 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.545892 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.548159 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.548161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.548161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.548169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.549876 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.550276 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.550277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.550278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.550339 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.550550 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.550551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.550551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.550670 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.550725 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.550899 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.550962 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260925 14:44:55.551173 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.541214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.193:37913)
0925 14:44:55.541413 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.541446 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.541922 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.542245 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.542250 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.542645 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.542946 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.542953 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.544007 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.544008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.545419 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.545423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.545919 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.545920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.546078 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.546588 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.546611 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.548051 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.549963 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.549970 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.549972 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.550196 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.550402 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.550403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.550403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.550495 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.550714 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.550717 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.550814 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.550885 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.550990 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260925 14:44:55.551659 23670 heartbeater.cc:347] Connected to a master server at 127.21.205.254:39085
I20260925 14:44:55.551746 23670 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:55.551934 23670 heartbeater.cc:511] Master 127.21.205.254:39085 requested a full tablet report, sending...
I20260925 14:44:55.552498 23457 ts_manager.cc:194] Registered new tserver with Master: 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945)
I20260925 14:44:55.553213 23457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:37913
I20260925 14:44:55.557000 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23531
I20260925 14:44:55.557124 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:44:55.557310 22327 external_mini_cluster.cc:1428] Setting key 22848efddde2246b104336e60c23c11b
I20260925 14:44:55.558167 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.552841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:44867 (local address 127.21.205.254:39085)
0925 14:44:55.552900 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.552902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.552964 (+ 62us) server_negotiation.cc:407] Connection header received
0925 14:44:55.553045 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.553046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.553056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.553111 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.553476 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.553957 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.554681 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.554896 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.554986 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.555888 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.555890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.555890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.555896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.557023 (+ 1127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.557399 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.557400 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.557400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.557453 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.557660 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.557661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.557662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.557776 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.557850 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.558013 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.558085 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:44:55.558183 23676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.552724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.193:44867)
0925 14:44:55.552887 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.552899 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.552982 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.553115 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.553116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.553295 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.553399 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.553401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.554059 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.554060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.554577 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.554578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.554988 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.554989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.555067 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.555478 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.555482 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.555744 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.557106 (+ 1362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.557107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.557108 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.557324 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.557529 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.557531 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.557531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.557589 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.557844 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.557845 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.557905 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.557988 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.558083 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:44:55.579932 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:39085
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:55.657537 23679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:55.657699 23679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:55.657721 23679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:55.658841 23679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:55.658879 23679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:55.658900 23679 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:55.658955 23679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:44:55.660880 23679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:55.661096 23679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:55.661345 23679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:55.666105 23685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:55.666112 23686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:55.666122 23688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:55.666276 23679 server_base.cc:1034] running on GCE node
I20260925 14:44:55.666440 23679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:55.666680 23679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:55.667851 23679 hybrid_clock.cc:648] HybridClock initialized: now 1790347495667844 us; error 36 us; skew 500 ppm
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:55.670198 23679 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:44:55.670635 23679 webserver.cc:460] Webserver started at http://127.21.205.194:42447/ using document root <none> and password file <none>
I20260925 14:44:55.670867 23679 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:55.670933 23679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:55.671061 23679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:55.672109 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "4e515303b6e64fb78028ee96ec4d5992"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "0ab0a9f818a53b4a2bf252b3bd151139"
server_key_iv: "824b2e6abd0a4c34fe799e3e1c5ecd32"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.672463 23679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "4e515303b6e64fb78028ee96ec4d5992"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "0ab0a9f818a53b4a2bf252b3bd151139"
server_key_iv: "824b2e6abd0a4c34fe799e3e1c5ecd32"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.673755 23679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:55.674477 23695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.674666 23679 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:55.674749 23679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "4e515303b6e64fb78028ee96ec4d5992"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "0ab0a9f818a53b4a2bf252b3bd151139"
server_key_iv: "824b2e6abd0a4c34fe799e3e1c5ecd32"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.674829 23679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:55.690117 23679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:55.692085 23679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:55.692230 23679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:55.692721 23679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:55.692780 23679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.692819 23679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:55.692845 23679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:55.699970 23679 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:43961
I20260925 14:44:55.700064 23811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:43961 every 8 connection(s)
I20260925 14:44:55.700429 23679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:44:55.702380 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.693089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:34451 (local address 127.21.205.254:39085)
0925 14:44:55.693135 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.693138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.693336 (+ 198us) server_negotiation.cc:407] Connection header received
0925 14:44:55.693836 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.693838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.693850 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.693933 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.694573 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.695378 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.697171 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.697374 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.697445 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.699863 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.699864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.699865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.699872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.701327 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.701659 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.701660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.701660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.701704 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.701902 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.701903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.701903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.702015 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.702064 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.702264 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.702319 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:44:55.702576 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.693071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.194:34451)
0925 14:44:55.693208 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.693244 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.693741 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.693979 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.693982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.694307 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.694478 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.694483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.695518 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.695519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.697065 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.697068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.697448 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.697449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.697612 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.698158 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.698168 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.699746 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.701393 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.701397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.701399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.701589 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.701758 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.701758 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.701758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.701847 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.702070 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.702074 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.702175 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.702251 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.702355 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":76,"threads_started":1}
I20260925 14:44:55.703071 23812 heartbeater.cc:347] Connected to a master server at 127.21.205.254:39085
I20260925 14:44:55.703151 23812 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:55.703332 23812 heartbeater.cc:511] Master 127.21.205.254:39085 requested a full tablet report, sending...
I20260925 14:44:55.703728 23458 ts_manager.cc:194] Registered new tserver with Master: 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194:43961)
I20260925 14:44:55.703948 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23679
I20260925 14:44:55.704034 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:44:55.704187 22327 external_mini_cluster.cc:1428] Setting key 209a83d2328f116001d87899973f3b13
I20260925 14:44:55.704315 23458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:34451
I20260925 14:44:55.710505 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.704504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:35415 (local address 127.21.205.254:39085)
0925 14:44:55.704542 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.704544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.704680 (+ 136us) server_negotiation.cc:407] Connection header received
0925 14:44:55.704724 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.704725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.704735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.704795 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.705139 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.705858 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.706828 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.706998 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.707049 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.707976 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.707977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.707977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.707982 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.709408 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.709808 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.709809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.709809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.709852 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.710049 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.710050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.710050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.710164 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.710211 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.710391 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.710440 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:44:55.710608 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.704479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.194:35415)
0925 14:44:55.704600 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.704609 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.704671 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.704799 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.704800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.704992 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.705057 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.705059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.705950 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.705951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.706744 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.706747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.707071 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.707072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.707154 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.707503 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.707507 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.707902 (+ 395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.709492 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.709493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.709494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.709726 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.709915 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.709916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.709916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.709985 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.710230 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.710231 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.710282 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.710378 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.710491 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:44:55.728780 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:39085
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:55.809288 23821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:55.809481 23821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:55.809514 23821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:55.810681 23821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:55.810745 23821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:55.810778 23821 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:55.810818 23821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:44:55.812263 23821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:39085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:55.812520 23821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:55.812740 23821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:55.817463 23829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:55.817585 23821 server_base.cc:1034] running on GCE node
W20260925 14:44:55.817485 23827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:55.817675 23826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:55.817907 23821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:55.818115 23821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:55.819262 23821 hybrid_clock.cc:648] HybridClock initialized: now 1790347495819253 us; error 25 us; skew 500 ppm
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:55.821745 23821 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:44:55.822211 23821 webserver.cc:460] Webserver started at http://127.21.205.195:43289/ using document root <none> and password file <none>
I20260925 14:44:55.822420 23821 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:55.822485 23821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:55.822629 23821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:55.823598 23821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "907ecc031d68401fa4115b2d262504b7"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "e3d9e1526558062097a73eb7d12b6ae5"
server_key_iv: "d760058bced1b162c832439d352f1abb"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.823958 23821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "907ecc031d68401fa4115b2d262504b7"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "e3d9e1526558062097a73eb7d12b6ae5"
server_key_iv: "d760058bced1b162c832439d352f1abb"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.825201 23821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:44:55.825815 23836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.826000 23821 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:44:55.826086 23821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "907ecc031d68401fa4115b2d262504b7"
format_stamp: "Formatted at 2026-09-25 14:44:55 on dist-test-slave-d7lw"
server_key: "e3d9e1526558062097a73eb7d12b6ae5"
server_key_iv: "d760058bced1b162c832439d352f1abb"
server_key_version: "encryptionkey@0"
I20260925 14:44:55.826162 23821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:55.836131 23821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:55.837697 23821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:55.837845 23821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:55.838337 23821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:55.838397 23821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:55.838431 23821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:55.838459 23821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:55.845652 23821 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:44899
I20260925 14:44:55.845742 23952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:44899 every 8 connection(s)
I20260925 14:44:55.846004 23821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:44:55.847888 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.838660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:42675 (local address 127.21.205.254:39085)
0925 14:44:55.838741 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.838745 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.838990 (+ 245us) server_negotiation.cc:407] Connection header received
0925 14:44:55.839363 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.839365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.839380 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.839461 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.840148 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.840972 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.841992 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.842206 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.842296 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.845329 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.845330 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.845331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.845338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.846830 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.847165 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.847166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.847166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.847215 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.847403 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.847403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.847403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.847490 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.847533 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.847706 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.847761 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 14:44:55.848114 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.838659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.195:42675)
0925 14:44:55.838850 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.838898 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.839271 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.839502 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.839505 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.839764 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.840056 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.840061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.841061 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.841062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.841880 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.841882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.842307 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.842308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.842445 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.842979 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.842998 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.845226 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.846900 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.846904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.846905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.847097 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.847277 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.847277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.847278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.847364 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.847524 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.847526 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.847618 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.847694 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.847904 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":126,"threads_started":1}
I20260925 14:44:55.848577 23953 heartbeater.cc:347] Connected to a master server at 127.21.205.254:39085
I20260925 14:44:55.848652 23953 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:55.848810 23953 heartbeater.cc:511] Master 127.21.205.254:39085 requested a full tablet report, sending...
I20260925 14:44:55.849182 23457 ts_manager.cc:194] Registered new tserver with Master: 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899)
I20260925 14:44:55.849653 23457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:42675
I20260925 14:44:55.852810 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 23821
I20260925 14:44:55.852891 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:44:55.853096 22327 external_mini_cluster.cc:1428] Setting key c9f3cb784f722c0abd8d149dfb0140cf
I20260925 14:44:55.853962 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:44:55.854796 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.849538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:56797 (local address 127.21.205.254:39085)
0925 14:44:55.849573 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.849575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.849617 (+ 42us) server_negotiation.cc:407] Connection header received
0925 14:44:55.849682 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.849683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.849690 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.849768 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.850114 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.850549 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.851227 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.851389 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.851466 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.852337 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.852338 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.852338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.852343 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.853877 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.854204 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.854205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.854205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.854251 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.854420 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.854421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.854421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.854497 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.854539 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.854699 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.854747 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:44:55.854975 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.849448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.21.205.195:56797)
0925 14:44:55.849565 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.849575 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.849630 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.849776 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.849777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.849986 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:55.850048 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.850049 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.850647 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.850647 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.851167 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.851168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.851490 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.851490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.851554 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.851936 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:55.851940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:55.852273 (+ 333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:55.853947 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.853948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.853949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.854139 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.854307 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:55.854308 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:55.854308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:55.854363 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:55.854552 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:55.854553 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:55.854595 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.854673 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.854784 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 14:44:55 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:55.875593 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.869027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.21.205.254:39085)
0925 14:44:55.869083 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.869085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.869160 (+ 75us) server_negotiation.cc:407] Connection header received
0925 14:44:55.869239 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.869240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.869250 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.869293 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:55.869852 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.870365 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.871081 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.871258 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.871335 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.873337 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:55.873339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:55.873349 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:55.873356 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:55.874457 (+ 1101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.874800 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.874801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.874801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.874886 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:55.875077 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:55.875078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:55.875078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:55.875182 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:55.875283 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.875428 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.875499 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:44:55.877024 23457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:44:55.878651 23457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:55.884698 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.880846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43961 (local address 127.0.0.1:55252)
0925 14:44:55.881000 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.881039 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.881101 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.881311 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.881313 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.881315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:55.881372 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.881373 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.882442 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.882443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.883389 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.883391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.884402 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.884403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.884480 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.884493 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.884565 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.884612 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260925 14:44:55.884733 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.880979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55252 (local address 127.21.205.194:43961)
0925 14:44:55.881154 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.881159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.881168 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:55.881206 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.881208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.881223 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.881309 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:55.881463 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.882359 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.883571 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.884324 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.884392 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.884445 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.884590 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.884642 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":71,"threads_started":1}
I20260925 14:44:55.885352 23973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.881183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:33945 (local address 127.0.0.1:52830)
0925 14:44:55.881389 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.881398 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.881439 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.881787 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.881787 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.881789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:55.881826 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.881828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.883079 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.883080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.884109 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.884112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.885100 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.885101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.885180 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.885184 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.885240 (+ 56us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.885278 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":25,"thread_start_us":58,"threads_started":1}
I20260925 14:44:55.885427 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.881379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.21.205.193:33945)
0925 14:44:55.881550 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.881554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.881564 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:44:55.881600 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.881602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.881617 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.881704 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:55.881883 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.882989 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.884195 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.884938 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.885000 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.885054 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.885279 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.885332 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260925 14:44:55.885720 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.881029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:44899 (local address 127.0.0.1:56984)
0925 14:44:55.881279 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.881291 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.881575 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.881824 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.881825 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.881827 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:55.881869 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.881870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.883087 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.883088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.884207 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.884210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.885398 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.885399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.885547 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.885551 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.885612 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.885652 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":24,"thread_start_us":62,"threads_started":1}
I20260925 14:44:55.885576 23745 tablet_service.cc:1467] Processing CreateTablet for tablet c569b5f44cf346909f5d522a126e63d6 (DEFAULT_TABLE table=test-table [id=92d7949502da46afbb94b300d2a55f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:55.885787 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.881109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56984 (local address 127.21.205.195:44899)
0925 14:44:55.881256 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.881260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.881542 (+ 282us) server_negotiation.cc:407] Connection header received
0925 14:44:55.881668 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.881670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.881702 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.881770 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:55.881929 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.882988 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.884301 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.885236 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.885306 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.885364 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.885639 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.885678 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260925 14:44:55.885944 23745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c569b5f44cf346909f5d522a126e63d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:55.888379 23871 tablet_service.cc:1467] Processing CreateTablet for tablet c569b5f44cf346909f5d522a126e63d6 (DEFAULT_TABLE table=test-table [id=92d7949502da46afbb94b300d2a55f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:55.888491 23601 tablet_service.cc:1467] Processing CreateTablet for tablet c569b5f44cf346909f5d522a126e63d6 (DEFAULT_TABLE table=test-table [id=92d7949502da46afbb94b300d2a55f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:55.888662 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c569b5f44cf346909f5d522a126e63d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:55.888768 23977 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Bootstrap starting.
I20260925 14:44:55.888810 23601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c569b5f44cf346909f5d522a126e63d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:55.889392 23977 tablet_bootstrap.cc:654] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:55.889701 23977 log.cc:824] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:55.890498 23977 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: No bootstrap required, opened a new log
I20260925 14:44:55.890549 23977 ts_tablet_manager.cc:1397] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:55.891623 23979 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Bootstrap starting.
I20260925 14:44:55.892256 23979 tablet_bootstrap.cc:654] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:55.892546 23979 log.cc:824] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:55.892565 23977 raft_consensus.cc:348] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.892755 23977 raft_consensus.cc:374] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:55.892807 23977 raft_consensus.cc:729] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Initialized, Role: FOLLOWER
I20260925 14:44:55.892935 23977 consensus_queue.cc:260] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.893231 23977 ts_tablet_manager.cc:1428] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:44:55.893297 23979 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: No bootstrap required, opened a new log
I20260925 14:44:55.893287 23812 heartbeater.cc:503] Master 127.21.205.254:39085 was elected leader, sending a full tablet report...
I20260925 14:44:55.893425 23979 ts_tablet_manager.cc:1397] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:55.893460 23980 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Bootstrap starting.
I20260925 14:44:55.894279 23980 tablet_bootstrap.cc:654] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:55.894590 23980 log.cc:824] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:55.895233 23980 tablet_bootstrap.cc:492] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: No bootstrap required, opened a new log
I20260925 14:44:55.895300 23980 ts_tablet_manager.cc:1397] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:55.895298 23979 raft_consensus.cc:348] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.895438 23979 raft_consensus.cc:374] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:55.895483 23979 raft_consensus.cc:729] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a40bbe2c8f4a1b924e990cc836be96, State: Initialized, Role: FOLLOWER
I20260925 14:44:55.895578 23979 consensus_queue.cc:260] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.895853 23979 ts_tablet_manager.cc:1428] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:55.895900 23670 heartbeater.cc:503] Master 127.21.205.254:39085 was elected leader, sending a full tablet report...
I20260925 14:44:55.897099 23980 raft_consensus.cc:348] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.897208 23980 raft_consensus.cc:374] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:55.897228 23980 raft_consensus.cc:729] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907ecc031d68401fa4115b2d262504b7, State: Initialized, Role: FOLLOWER
I20260925 14:44:55.897331 23980 consensus_queue.cc:260] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.897562 23980 ts_tablet_manager.cc:1428] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:55.897583 23953 heartbeater.cc:503] Master 127.21.205.254:39085 was elected leader, sending a full tablet report...
W20260925 14:44:55.951136 23813 tablet.cc:2369] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:44:55.992432 23982 raft_consensus.cc:482] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:55.992559 23982 raft_consensus.cc:504] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.992885 23982 leader_election.cc:288] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899), 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945)
I20260925 14:44:55.995644 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.993061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:44899 (local address 127.21.205.194:35473)
0925 14:44:55.993114 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.993126 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.993170 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.993368 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.993369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.993371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:55.993465 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.993466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.994094 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.994095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.994851 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.994853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.995404 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.995405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.995481 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.995484 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.995541 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.995577 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:44:55.995646 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.993126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:35473 (local address 127.21.205.195:44899)
0925 14:44:55.993205 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.993207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.993218 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:44:55.993256 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.993257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.993266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.993364 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:55.993538 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.994014 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.994940 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.995315 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.995393 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.995403 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.995559 (+ 156us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.995593 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:44:55.996045 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.993300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:47031 (local address 127.21.205.193:33945)
0925 14:44:55.993360 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:55.993362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:55.993472 (+ 110us) server_negotiation.cc:407] Connection header received
0925 14:44:55.993531 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:55.993532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:55.993541 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:55.993584 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:55.993747 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.994200 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.995188 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.995597 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.995702 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.995711 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:55.995924 (+ 213us) server_negotiation.cc:299] Negotiation successful
0925 14:44:55.995977 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:44:55.995999 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c569b5f44cf346909f5d522a126e63d6" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7" is_pre_election: true
I20260925 14:44:55.996150 23906 raft_consensus.cc:2454] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:55.996274 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:55.993209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:33945 (local address 127.21.205.194:47031)
0925 14:44:55.993373 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:55.993400 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:55.993477 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:55.993596 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:55.993596 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:55.993598 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:55.993679 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.993680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.994281 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.994282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.995093 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:55.995095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:55.995724 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:55.995725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:55.995837 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:55.995842 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:44:55.996149 (+ 307us) client_negotiation.cc:240] Negotiation successful
0925 14:44:55.996190 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":96,"threads_started":1}
I20260925 14:44:55.996413 23699 leader_election.cc:302] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e515303b6e64fb78028ee96ec4d5992, 907ecc031d68401fa4115b2d262504b7; no voters:
I20260925 14:44:55.996543 23982 raft_consensus.cc:2781] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:55.996625 23982 raft_consensus.cc:482] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:55.996649 23982 raft_consensus.cc:3037] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:55.996779 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c569b5f44cf346909f5d522a126e63d6" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96" is_pre_election: true
I20260925 14:44:55.996928 23622 raft_consensus.cc:2454] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:55.997658 23982 raft_consensus.cc:504] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.997768 23982 leader_election.cc:288] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Requested vote from peers 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899), 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945)
I20260925 14:44:55.997900 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c569b5f44cf346909f5d522a126e63d6" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7"
I20260925 14:44:55.997926 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c569b5f44cf346909f5d522a126e63d6" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96"
I20260925 14:44:55.997989 23906 raft_consensus.cc:3037] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:55.998009 23622 raft_consensus.cc:3037] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:55.998659 23906 raft_consensus.cc:2454] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:55.998778 23622 raft_consensus.cc:2454] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:55.998871 23699 leader_election.cc:302] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e515303b6e64fb78028ee96ec4d5992, 907ecc031d68401fa4115b2d262504b7; no voters:
I20260925 14:44:55.998971 23982 raft_consensus.cc:2781] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:55.999130 23982 raft_consensus.cc:686] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 LEADER]: Becoming Leader. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Running, Role: LEADER
I20260925 14:44:55.999228 23982 consensus_queue.cc:237] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:55.999963 23457 catalog_manager.cc:5495] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } health_report { overall_health: UNKNOWN } } }
I20260925 14:44:56.031937 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.029489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55256 (local address 127.21.205.194:43961)
0925 14:44:56.029554 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.029556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.029565 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:56.029674 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.029675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.029699 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.029781 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:56.029911 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.030392 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.031099 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.031308 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.031381 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.031581 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:56.031672 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:56.031752 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.031837 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.031856 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 14:44:56.033684 23622 raft_consensus.cc:1261] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.033684 23906 raft_consensus.cc:1261] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.034047 23987 consensus_queue.cc:1035] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.034174 23982 consensus_queue.cc:1035] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.036926 23992 mvcc.cc:205] Tried to move back new op lower bound from 7333263343751180288 to 7333263343613317120. Current Snapshot: MvccSnapshot[applied={T|T < 7333263343751180288}]
I20260925 14:44:56.039844 23457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:56.039947 23457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:56.040705 23457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Sending DeleteTablet for 3 replicas of tablet c569b5f44cf346909f5d522a126e63d6
I20260925 14:44:56.041014 23745 tablet_service.cc:1514] Processing DeleteTablet for tablet c569b5f44cf346909f5d522a126e63d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:55252
I20260925 14:44:56.041059 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet c569b5f44cf346909f5d522a126e63d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:56984
I20260925 14:44:56.041096 23601 tablet_service.cc:1514] Processing DeleteTablet for tablet c569b5f44cf346909f5d522a126e63d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:52830
I20260925 14:44:56.041169 23457 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 14:44:56.041311 23999 tablet_replica.cc:330] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: stopping tablet replica
W20260925 14:44:56.041333 23457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:56.041391 23999 raft_consensus.cc:2229] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:44:56.041488 23999 raft_consensus.cc:2258] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.041884 23999 ts_tablet_manager.cc:1905] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.042529 23745 tablet_service.cc:1467] Processing CreateTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 (DEFAULT_TABLE table=test-table [id=602d9d89908546ac8c1de741e30d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:56.042575 23871 tablet_service.cc:1467] Processing CreateTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 (DEFAULT_TABLE table=test-table [id=602d9d89908546ac8c1de741e30d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:56.042572 23601 tablet_service.cc:1467] Processing CreateTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 (DEFAULT_TABLE table=test-table [id=602d9d89908546ac8c1de741e30d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:56.042651 23745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a839ee9b2de248a7a31f0386a06aa3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:56.042663 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a839ee9b2de248a7a31f0386a06aa3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:56.042685 23601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a839ee9b2de248a7a31f0386a06aa3d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:56.043978 23980 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Bootstrap starting.
I20260925 14:44:56.043977 23977 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Bootstrap starting.
I20260925 14:44:56.044397 23977 tablet_bootstrap.cc:654] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.044528 23980 tablet_bootstrap.cc:654] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.044893 23979 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Bootstrap starting.
I20260925 14:44:56.045145 23977 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: No bootstrap required, opened a new log
I20260925 14:44:56.045202 23977 ts_tablet_manager.cc:1397] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:56.045377 23977 raft_consensus.cc:348] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.045452 23977 raft_consensus.cc:374] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.045475 23977 raft_consensus.cc:729] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.045524 23977 consensus_queue.cc:260] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.045578 23979 tablet_bootstrap.cc:654] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.045629 23977 ts_tablet_manager.cc:1428] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:56.046819 23999 ts_tablet_manager.cc:1918] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:44:56.046880 23999 log.cc:1196] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/c569b5f44cf346909f5d522a126e63d6
I20260925 14:44:56.046988 23979 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: No bootstrap required, opened a new log
I20260925 14:44:56.047029 23979 ts_tablet_manager.cc:1397] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:56.047082 23999 ts_tablet_manager.cc:1939] T c569b5f44cf346909f5d522a126e63d6 P 4e515303b6e64fb78028ee96ec4d5992: Deleting consensus metadata
I20260925 14:44:56.047178 23979 raft_consensus.cc:348] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.047240 23979 raft_consensus.cc:374] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.047260 23979 raft_consensus.cc:729] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a40bbe2c8f4a1b924e990cc836be96, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.047302 23979 consensus_queue.cc:260] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.047379 23979 ts_tablet_manager.cc:1428] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:56.047429 23445 catalog_manager.cc:4841] TS 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194:43961): tablet c569b5f44cf346909f5d522a126e63d6 (table test-table [id=92d7949502da46afbb94b300d2a55f3a]) successfully deleted
I20260925 14:44:56.047497 23980 tablet_bootstrap.cc:492] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: No bootstrap required, opened a new log
I20260925 14:44:56.047533 23980 ts_tablet_manager.cc:1397] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 14:44:56.047683 23980 raft_consensus.cc:348] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.047804 23980 raft_consensus.cc:374] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.047835 23980 raft_consensus.cc:729] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907ecc031d68401fa4115b2d262504b7, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.047891 23980 consensus_queue.cc:260] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.048003 23980 ts_tablet_manager.cc:1428] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260925 14:44:56.049594 23671 tablet.cc:2369] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:44:56.047984 24000 tablet_replica.cc:330] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: stopping tablet replica
I20260925 14:44:56.051935 24000 raft_consensus.cc:2229] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:56.052058 24000 raft_consensus.cc:2258] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.052214 24001 tablet_replica.cc:330] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: stopping tablet replica
I20260925 14:44:56.052300 24001 raft_consensus.cc:2229] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:56.052394 24001 raft_consensus.cc:2258] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.052503 24000 ts_tablet_manager.cc:1905] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.052861 24001 ts_tablet_manager.cc:1905] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.054044 24000 ts_tablet_manager.cc:1918] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:44:56.054064 24001 ts_tablet_manager.cc:1918] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:44:56.054112 24000 log.cc:1196] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/c569b5f44cf346909f5d522a126e63d6
I20260925 14:44:56.054136 24001 log.cc:1196] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/c569b5f44cf346909f5d522a126e63d6
I20260925 14:44:56.054361 24001 ts_tablet_manager.cc:1939] T c569b5f44cf346909f5d522a126e63d6 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting consensus metadata
I20260925 14:44:56.054371 24000 ts_tablet_manager.cc:1939] T c569b5f44cf346909f5d522a126e63d6 P 907ecc031d68401fa4115b2d262504b7: Deleting consensus metadata
I20260925 14:44:56.054713 23445 catalog_manager.cc:4841] TS 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945): tablet c569b5f44cf346909f5d522a126e63d6 (table test-table [id=92d7949502da46afbb94b300d2a55f3a]) successfully deleted
I20260925 14:44:56.054814 23445 catalog_manager.cc:4841] TS 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899): tablet c569b5f44cf346909f5d522a126e63d6 (table test-table [id=92d7949502da46afbb94b300d2a55f3a]) successfully deleted
I20260925 14:44:56.062898 23982 raft_consensus.cc:482] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:56.062970 23982 raft_consensus.cc:504] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.063104 23982 leader_election.cc:288] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945), 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899)
I20260925 14:44:56.063251 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a839ee9b2de248a7a31f0386a06aa3d9" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96" is_pre_election: true
I20260925 14:44:56.063267 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a839ee9b2de248a7a31f0386a06aa3d9" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7" is_pre_election: true
I20260925 14:44:56.063334 23906 raft_consensus.cc:2454] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:56.063334 23622 raft_consensus.cc:2454] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:56.063481 23699 leader_election.cc:302] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a40bbe2c8f4a1b924e990cc836be96, 4e515303b6e64fb78028ee96ec4d5992; no voters:
I20260925 14:44:56.063593 23982 raft_consensus.cc:2781] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:56.063653 23982 raft_consensus.cc:482] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:56.063685 23982 raft_consensus.cc:3037] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.064268 23982 raft_consensus.cc:504] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.064395 23982 leader_election.cc:288] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Requested vote from peers 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945), 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899)
I20260925 14:44:56.064535 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a839ee9b2de248a7a31f0386a06aa3d9" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7"
I20260925 14:44:56.064539 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a839ee9b2de248a7a31f0386a06aa3d9" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96"
I20260925 14:44:56.064610 23906 raft_consensus.cc:3037] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.064628 23622 raft_consensus.cc:3037] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.065183 23622 raft_consensus.cc:2454] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:56.065249 23906 raft_consensus.cc:2454] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:56.065403 23699 leader_election.cc:302] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e515303b6e64fb78028ee96ec4d5992, 907ecc031d68401fa4115b2d262504b7; no voters:
I20260925 14:44:56.065554 23982 raft_consensus.cc:2781] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:56.065616 23982 raft_consensus.cc:686] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 LEADER]: Becoming Leader. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Running, Role: LEADER
I20260925 14:44:56.065657 23982 consensus_queue.cc:237] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } }
I20260925 14:44:56.066109 23458 catalog_manager.cc:5495] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } health_report { overall_health: UNKNOWN } } }
Sep 25 14:44:56 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347494, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:56.092015 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.084414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39085 (local address 127.0.0.1:60800)
0925 14:44:56.084576 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:56.084600 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:56.084667 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:56.084829 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:56.084830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:56.085260 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:56.085315 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.085318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.085938 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.085939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.086810 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.086812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.087139 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.087140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.087200 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.087719 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:56.087724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:56.089269 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:56.090884 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:56.090892 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:56.090894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:56.091106 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:56.091295 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:56.091296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:56.091297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:56.091351 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:56.091588 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:56.091596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:56.091637 (+ 41us) client_negotiation.cc:769] Sending connection context
0925 14:44:56.091716 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 14:44:56.091857 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260925 14:44:56.092015 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.084488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.21.205.254:39085)
0925 14:44:56.084541 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.084543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.084661 (+ 118us) server_negotiation.cc:407] Connection header received
0925 14:44:56.084724 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.084725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.084733 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.084801 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:56.085388 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.085865 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.086883 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.087067 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.087112 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.089351 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:56.089352 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:56.089352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:56.089359 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:56.090795 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.091176 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.091177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.091178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.091232 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.091418 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.091419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.091419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.091517 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:56.091585 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.091737 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.091903 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":28}
I20260925 14:44:56.093990 23458 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:60800:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 14:44:56.094179 23458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:56.095661 23745 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9d3f122bde4ed3b7134892f8ddacce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa9a1da3c8d34e2abac51f0590461396]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:56.095844 23745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9d3f122bde4ed3b7134892f8ddacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:56.095835 23601 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9d3f122bde4ed3b7134892f8ddacce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa9a1da3c8d34e2abac51f0590461396]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:56.095966 23601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9d3f122bde4ed3b7134892f8ddacce. 1 dirs total, 0 dirs full, 0 dirs failed
W20260925 14:44:56.096514 23954 tablet.cc:2369] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:44:56.097225 23979 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: Bootstrap starting.
I20260925 14:44:56.097352 23871 tablet_service.cc:1467] Processing CreateTablet for tablet 8b9d3f122bde4ed3b7134892f8ddacce (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=aa9a1da3c8d34e2abac51f0590461396]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:56.097464 23871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b9d3f122bde4ed3b7134892f8ddacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:56.097587 23979 tablet_bootstrap.cc:654] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.098168 23979 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: No bootstrap required, opened a new log
I20260925 14:44:56.098233 23979 ts_tablet_manager.cc:1397] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:56.098398 23979 raft_consensus.cc:348] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.098474 23979 raft_consensus.cc:374] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.098502 23979 raft_consensus.cc:729] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43a40bbe2c8f4a1b924e990cc836be96, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.098541 23979 consensus_queue.cc:260] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.098626 23979 ts_tablet_manager.cc:1428] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:56.098803 23977 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: Bootstrap starting.
I20260925 14:44:56.099143 23977 tablet_bootstrap.cc:654] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.099507 23984 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.099591 23984 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.099670 23980 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: Bootstrap starting.
I20260925 14:44:56.100082 23980 tablet_bootstrap.cc:654] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:56.100497 23977 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: No bootstrap required, opened a new log
I20260925 14:44:56.100553 23977 ts_tablet_manager.cc:1397] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:56.100661 23980 tablet_bootstrap.cc:492] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: No bootstrap required, opened a new log
I20260925 14:44:56.100689 23977 raft_consensus.cc:348] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.100728 23980 ts_tablet_manager.cc:1397] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 14:44:56.100756 23977 raft_consensus.cc:374] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.100776 23977 raft_consensus.cc:729] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.100816 23977 consensus_queue.cc:260] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.100868 23980 raft_consensus.cc:348] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.100899 23977 ts_tablet_manager.cc:1428] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:56.100931 23980 raft_consensus.cc:374] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:56.100957 23980 raft_consensus.cc:729] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907ecc031d68401fa4115b2d262504b7, State: Initialized, Role: FOLLOWER
I20260925 14:44:56.101002 23980 consensus_queue.cc:260] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.100993 23982 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.101048 23982 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.101082 23980 ts_tablet_manager.cc:1428] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260925 14:44:56.101219 23985 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.101322 23985 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.180375 23982 raft_consensus.cc:482] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:56.180457 23982 raft_consensus.cc:504] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.180603 23982 leader_election.cc:288] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899), 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945)
I20260925 14:44:56.180819 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9d3f122bde4ed3b7134892f8ddacce" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96" is_pre_election: true
I20260925 14:44:56.180840 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9d3f122bde4ed3b7134892f8ddacce" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7" is_pre_election: true
I20260925 14:44:56.180915 23622 raft_consensus.cc:2454] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:56.180923 23906 raft_consensus.cc:2454] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 0.
I20260925 14:44:56.181077 23699 leader_election.cc:302] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e515303b6e64fb78028ee96ec4d5992, 907ecc031d68401fa4115b2d262504b7; no voters:
I20260925 14:44:56.181156 23982 raft_consensus.cc:2781] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:56.181203 23982 raft_consensus.cc:482] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:56.181218 23982 raft_consensus.cc:3037] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.181840 23982 raft_consensus.cc:504] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.181963 23982 leader_election.cc:288] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Requested vote from peers 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899), 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945)
I20260925 14:44:56.182116 23906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9d3f122bde4ed3b7134892f8ddacce" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907ecc031d68401fa4115b2d262504b7"
I20260925 14:44:56.182150 23622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b9d3f122bde4ed3b7134892f8ddacce" candidate_uuid: "4e515303b6e64fb78028ee96ec4d5992" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43a40bbe2c8f4a1b924e990cc836be96"
I20260925 14:44:56.182211 23622 raft_consensus.cc:3037] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.182215 23906 raft_consensus.cc:3037] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:56.182821 23622 raft_consensus.cc:2454] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:56.182823 23906 raft_consensus.cc:2454] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e515303b6e64fb78028ee96ec4d5992 in term 1.
I20260925 14:44:56.182978 23699 leader_election.cc:302] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43a40bbe2c8f4a1b924e990cc836be96, 4e515303b6e64fb78028ee96ec4d5992; no voters:
I20260925 14:44:56.183115 23982 raft_consensus.cc:2781] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:56.183209 23982 raft_consensus.cc:686] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [term 1 LEADER]: Becoming Leader. State: Replica: 4e515303b6e64fb78028ee96ec4d5992, State: Running, Role: LEADER
I20260925 14:44:56.183269 23982 consensus_queue.cc:237] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } }
I20260925 14:44:56.183542 23987 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: TxnStatusTablet state changed. Reason: New leader 4e515303b6e64fb78028ee96ec4d5992. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.183614 23987 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:56.183821 24016 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 14:44:56.183910 24016 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 14:44:56.183900 23457 catalog_manager.cc:5495] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } health_report { overall_health: UNKNOWN } } }
I20260925 14:44:56.196626 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.194045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.21.205.194:43961)
0925 14:44:56.194094 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.194096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.194105 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:56.194188 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.194189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.194200 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.194281 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:56.194409 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.194993 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.195911 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.196174 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.196250 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.196410 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:56.196441 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:56.196490 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.196543 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.196565 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:44:56.196656 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.193993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43961 (local address 127.0.0.1:55272)
0925 14:44:56.194058 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:56.194070 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:56.194120 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:56.194294 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:56.194295 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:56.194297 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:56.194344 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.194346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.195072 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.195073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.195775 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.195778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.196242 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.196243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.196304 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.196326 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:56.196494 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:56.196500 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 14:44:56.196543 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 14:44:56.196585 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 14:44:56.197516 23622 raft_consensus.cc:1261] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.197523 23906 raft_consensus.cc:1261] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.197696 23987 consensus_queue.cc:1035] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.197804 23987 consensus_queue.cc:1035] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.198228 23984 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: TxnStatusTablet state changed. Reason: New leader 4e515303b6e64fb78028ee96ec4d5992. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.198315 23984 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.198350 23985 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: TxnStatusTablet state changed. Reason: New leader 4e515303b6e64fb78028ee96ec4d5992. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.198477 23985 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.198864 23987 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.198952 23987 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:56.199327 23991 mvcc.cc:205] Tried to move back new op lower bound from 7333263344423624704 to 7333263344367149056. Current Snapshot: MvccSnapshot[applied={T|T < 7333263344423624704}]
I20260925 14:44:56.199451 23984 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.199515 23984 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 43a40bbe2c8f4a1b924e990cc836be96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.199916 23987 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.199987 23987 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 4e515303b6e64fb78028ee96ec4d5992: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:56.200196 23985 tablet_replica.cc:439] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4e515303b6e64fb78028ee96ec4d5992" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e515303b6e64fb78028ee96ec4d5992" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43961 } } peers { permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 } } peers { permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 } } }
I20260925 14:44:56.200268 23985 tablet_replica.cc:442] T 8b9d3f122bde4ed3b7134892f8ddacce P 907ecc031d68401fa4115b2d262504b7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:56.204457 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.201985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:53651 (local address 127.21.205.194:43961)
0925 14:44:56.202016 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.202017 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.202046 (+ 29us) server_negotiation.cc:407] Connection header received
0925 14:44:56.202116 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.202117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.202124 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.202158 (+ 34us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:56.202259 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.202719 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.203847 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.204074 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.204134 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.204315 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:56.204335 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:56.204369 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.204408 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.204421 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 14:44:56.204564 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.201940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43961 (local address 127.21.205.194:53651)
0925 14:44:56.202006 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:56.202016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:56.202059 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:56.202164 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:56.202165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:56.202166 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:56.202206 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.202207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.202816 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.202817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.203714 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:56.203716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.204152 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.204153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:56.204211 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.204225 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:56.204370 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:56.204372 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:44:56.204409 (+ 37us) client_negotiation.cc:240] Negotiation successful
0925 14:44:56.204447 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:44:56.206826 23906 raft_consensus.cc:1261] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.206839 23622 raft_consensus.cc:1261] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Refusing update from remote peer 4e515303b6e64fb78028ee96ec4d5992: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:56.207042 24022 consensus_queue.cc:1035] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907ecc031d68401fa4115b2d262504b7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 44899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.207127 23982 consensus_queue.cc:1035] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43a40bbe2c8f4a1b924e990cc836be96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:56.208405 23990 mvcc.cc:205] Tried to move back new op lower bound from 7333263344462045184 to 7333263343885271040. Current Snapshot: MvccSnapshot[applied={T|T < 7333263344462045184}]
I20260925 14:44:56.219864 23457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:56.219944 23457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:56.220528 23457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7c4eed5a3ca44f9383456694a04baec2: Sending DeleteTablet for 3 replicas of tablet a839ee9b2de248a7a31f0386a06aa3d9
I20260925 14:44:56.220741 23601 tablet_service.cc:1514] Processing DeleteTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:52830
I20260925 14:44:56.220822 24001 tablet_replica.cc:330] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: stopping tablet replica
I20260925 14:44:56.220857 23744 tablet_service.cc:1514] Processing DeleteTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:55252
I20260925 14:44:56.220884 24001 raft_consensus.cc:2229] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:56.220917 24001 raft_consensus.cc:2258] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.220940 23999 tablet_replica.cc:330] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: stopping tablet replica
I20260925 14:44:56.220994 23999 raft_consensus.cc:2229] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:44:56.221043 23999 raft_consensus.cc:2258] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.221307 23999 ts_tablet_manager.cc:1905] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.221616 23871 tablet_service.cc:1514] Processing DeleteTablet for tablet a839ee9b2de248a7a31f0386a06aa3d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:56 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:56984
I20260925 14:44:56.221692 24000 tablet_replica.cc:330] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: stopping tablet replica
I20260925 14:44:56.221731 24000 raft_consensus.cc:2229] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:56.221758 24000 raft_consensus.cc:2258] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:56.222033 24001 ts_tablet_manager.cc:1905] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.222868 23999 ts_tablet_manager.cc:1918] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:56.222930 23999 log.cc:1196] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/a839ee9b2de248a7a31f0386a06aa3d9
I20260925 14:44:56.223155 23999 ts_tablet_manager.cc:1939] T a839ee9b2de248a7a31f0386a06aa3d9 P 4e515303b6e64fb78028ee96ec4d5992: Deleting consensus metadata
I20260925 14:44:56.223480 23445 catalog_manager.cc:4841] TS 4e515303b6e64fb78028ee96ec4d5992 (127.21.205.194:43961): tablet a839ee9b2de248a7a31f0386a06aa3d9 (table test-table [id=602d9d89908546ac8c1de741e30d19f0]) successfully deleted
I20260925 14:44:56.223615 24001 ts_tablet_manager.cc:1918] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:56.223660 24001 log.cc:1196] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/a839ee9b2de248a7a31f0386a06aa3d9
I20260925 14:44:56.223664 24000 ts_tablet_manager.cc:1905] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:56.223943 24001 ts_tablet_manager.cc:1939] T a839ee9b2de248a7a31f0386a06aa3d9 P 43a40bbe2c8f4a1b924e990cc836be96: Deleting consensus metadata
I20260925 14:44:56.224206 23445 catalog_manager.cc:4841] TS 43a40bbe2c8f4a1b924e990cc836be96 (127.21.205.193:33945): tablet a839ee9b2de248a7a31f0386a06aa3d9 (table test-table [id=602d9d89908546ac8c1de741e30d19f0]) successfully deleted
I20260925 14:44:56.225023 24000 ts_tablet_manager.cc:1918] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:56.225082 24000 log.cc:1196] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/a839ee9b2de248a7a31f0386a06aa3d9
I20260925 14:44:56.225270 24000 ts_tablet_manager.cc:1939] T a839ee9b2de248a7a31f0386a06aa3d9 P 907ecc031d68401fa4115b2d262504b7: Deleting consensus metadata
I20260925 14:44:56.225621 23445 catalog_manager.cc:4841] TS 907ecc031d68401fa4115b2d262504b7 (127.21.205.195:44899): tablet a839ee9b2de248a7a31f0386a06aa3d9 (table test-table [id=602d9d89908546ac8c1de741e30d19f0]) successfully deleted
Sep 25 14:44:56 dist-test-slave-d7lw krb5kdc[23413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347495, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:44:56.232533 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.223482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52832 (local address 127.21.205.193:33945)
0925 14:44:56.223569 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.223571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.223578 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:44:56.223597 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.223598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.223606 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.223658 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:56.224594 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.225294 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.226276 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.226468 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.226536 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.229720 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:56.229727 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:56.229728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:56.229734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:56.231298 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.231659 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.231662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.231663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.231732 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.231996 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.231997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.231998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.232145 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:56.232296 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.232370 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.232441 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260925 14:44:56.232852 23664 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52832
I20260925 14:44:56.239308 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:56.233760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.21.205.254:39085)
0925 14:44:56.233801 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:56.233803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:56.233861 (+ 58us) server_negotiation.cc:407] Connection header received
0925 14:44:56.233933 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:56.233933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:56.233942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:56.233996 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:56.234470 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.234936 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.235614 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:56.235887 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:56.235973 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:56.237041 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:56.237042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:56.237043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:56.237049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:56.238273 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.238597 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.238598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.238598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.238651 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:56.238842 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:56.238842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:56.238843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:56.238948 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:56.239045 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:56.239145 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 14:44:56.239215 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 14:44:56.239634 23457 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60806
I20260925 14:44:56.240266 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23531
I20260925 14:44:56.245828 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23679
I20260925 14:44:56.252094 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23821
I20260925 14:44:56.256817 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 23426
2026-09-25T14:44:56Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5907 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:44:56 dist-test-slave-d7lw krb5kdc[24033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:44:56 dist-test-slave-d7lw krb5kdc[24033](info): set up 2 sockets
Sep 25 14:44:56 dist-test-slave-d7lw krb5kdc[24033](info): commencing operation
krb5kdc: starting...
W20260925 14:44:57.274410 22327 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:44:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:44:57.380663 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38521
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:38521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:57.458976 24046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:57.459151 24046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:57.459172 24046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:57.460372 24046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:44:57.460412 24046 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 14:44:57.460425 24046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:57.460438 24046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:57.460475 24046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:44:57.460500 24046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:44:57.461915 24046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:38521
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:57.462152 24046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:57.462370 24046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:57.466971 24051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:57.467121 24046 server_base.cc:1034] running on GCE node
W20260925 14:44:57.466964 24052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:57.466964 24054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:57.467393 24046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:57.467617 24046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:57.468771 24046 hybrid_clock.cc:648] HybridClock initialized: now 1790347497468754 us; error 41 us; skew 500 ppm
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:57.471084 24046 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:44:57.471475 24046 webserver.cc:460] Webserver started at http://127.21.205.254:39097/ using document root <none> and password file <none>
I20260925 14:44:57.471693 24046 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:57.471736 24046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:57.471905 24046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:57.472909 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "79f7e1ab9e8944c3a92f77b5d815212f"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "1038838437165af18c1c559fa324ffc8"
server_key_iv: "8784977f833724744384116fb8cfc91c"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.473261 24046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "79f7e1ab9e8944c3a92f77b5d815212f"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "1038838437165af18c1c559fa324ffc8"
server_key_iv: "8784977f833724744384116fb8cfc91c"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.474607 24046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:44:57.475299 24062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:57.475481 24046 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:44:57.475577 24046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "79f7e1ab9e8944c3a92f77b5d815212f"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "1038838437165af18c1c559fa324ffc8"
server_key_iv: "8784977f833724744384116fb8cfc91c"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.475684 24046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:57.497217 24046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:57.498875 24046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:57.499032 24046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:57.503545 24046 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:38521
I20260925 14:44:57.503679 24124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:38521 every 8 connection(s)
I20260925 14:44:57.503973 24046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:44:57.504457 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24046
I20260925 14:44:57.504527 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:44:57.504681 24125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:57.504706 22327 external_mini_cluster.cc:1428] Setting key 3a12a9ae1d3c70dba6367fb5890ed5e2
I20260925 14:44:57.508185 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Bootstrap starting.
I20260925 14:44:57.508893 24125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:57.509215 24125 log.cc:824] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:57.510304 24125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: No bootstrap required, opened a new log
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:57.513010 24125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } }
I20260925 14:44:57.513224 24125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:57.513270 24125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79f7e1ab9e8944c3a92f77b5d815212f, State: Initialized, Role: FOLLOWER
I20260925 14:44:57.513423 24125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } }
I20260925 14:44:57.513520 24125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:44:57.513558 24125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:44:57.513602 24125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:57.514334 24125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } }
I20260925 14:44:57.514458 24125 leader_election.cc:302] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79f7e1ab9e8944c3a92f77b5d815212f; no voters:
I20260925 14:44:57.514621 24125 leader_election.cc:288] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:44:57.514669 24130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:57.514809 24130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [term 1 LEADER]: Becoming Leader. State: Replica: 79f7e1ab9e8944c3a92f77b5d815212f, State: Running, Role: LEADER
I20260925 14:44:57.514941 24125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:44:57.514971 24130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } }
I20260925 14:44:57.515282 24130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79f7e1ab9e8944c3a92f77b5d815212f. Latest consensus state: current_term: 1 leader_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } } }
I20260925 14:44:57.515439 24130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:57.515640 24131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79f7e1ab9e8944c3a92f77b5d815212f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38521 } } }
I20260925 14:44:57.515712 24131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f [sys.catalog]: This master's current role is: LEADER
I20260925 14:44:57.515929 24139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:44:57.515942 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.505308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.21.205.254:38521)
0925 14:44:57.505467 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:57.505474 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:57.505496 (+ 22us) server_negotiation.cc:407] Connection header received
0925 14:44:57.505872 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:57.505878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:57.505919 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:57.506021 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:57.506826 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.507735 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.508837 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.509289 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.509452 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.512128 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:57.512133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:57.512133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:57.512145 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:57.514126 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.514481 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.514484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.514484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.514607 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.514883 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.514884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.514884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.515195 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:57.515363 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:57.515472 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 14:44:57.515602 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":72,"threads_started":1}
I20260925 14:44:57.516794 24139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:44:57.518240 24139 catalog_manager.cc:1347] Generated new cluster ID: 24e99f559f83418f93398c7a5b51def8
I20260925 14:44:57.518280 24139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:44:57.549899 24139 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:44:57.550393 24139 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:44:57.555219 24139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Generated new TSK 0
I20260925 14:44:57.555375 24139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:44:57.590464 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:38521
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:57.695515 24151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:57.695703 24151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:57.695727 24151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:57.696798 24151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:57.696832 24151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:57.696851 24151 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 14:44:57.696862 24151 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:57.696878 24151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:44:57.698161 24151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:38521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:57.698325 24151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:57.698498 24151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:57.703195 24156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:57.703222 24157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:57.703293 24151 server_base.cc:1034] running on GCE node
W20260925 14:44:57.703348 24159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:57.703579 24151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:57.703816 24151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:57.704986 24151 hybrid_clock.cc:648] HybridClock initialized: now 1790347497704967 us; error 37 us; skew 500 ppm
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:57.707233 24151 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:44:57.707563 24151 webserver.cc:460] Webserver started at http://127.21.205.193:43755/ using document root <none> and password file <none>
I20260925 14:44:57.707762 24151 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:57.707862 24151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:57.707957 24151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:57.708869 24151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "a0825ee4b4394c0d84229620848b07ea"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "85f72ec87e22ce6ef91da750305cacb4"
server_key_iv: "815da7ba548edddd094bc25b49f72c0a"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.709152 24151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "a0825ee4b4394c0d84229620848b07ea"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "85f72ec87e22ce6ef91da750305cacb4"
server_key_iv: "815da7ba548edddd094bc25b49f72c0a"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.710312 24151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:57.710943 24166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:57.711102 24151 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:57.711179 24151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "a0825ee4b4394c0d84229620848b07ea"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "85f72ec87e22ce6ef91da750305cacb4"
server_key_iv: "815da7ba548edddd094bc25b49f72c0a"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.711230 24151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:57.737900 24151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:57.739573 24151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:57.739675 24151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:57.740159 24151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:57.740199 24151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:57.740222 24151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:57.740259 24151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:57.747188 24151 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:46425
I20260925 14:44:57.747252 24283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:46425 every 8 connection(s)
I20260925 14:44:57.747536 24151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:44:57.750970 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.740642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:44819 (local address 127.21.205.254:38521)
0925 14:44:57.740728 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:57.740730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:57.740870 (+ 140us) server_negotiation.cc:407] Connection header received
0925 14:44:57.741303 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:57.741304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:57.741316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:57.741400 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:57.742215 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.743054 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.744645 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.745015 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.745080 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.747867 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:57.747869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:57.747869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:57.747877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:57.749417 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.749883 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.749885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.749885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.749978 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.750254 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.750255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.750255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.750433 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:57.750493 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:57.750803 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 14:44:57.750883 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 14:44:57.751127 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.740570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.193:44819)
0925 14:44:57.740741 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:57.740778 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:57.741205 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:57.741446 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:57.741449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:57.741807 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:57.742085 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.742091 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.743149 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.743150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.744540 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.744544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.745163 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.745164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.745316 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.745855 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:57.745865 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:57.747715 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:57.749528 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.749534 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.749537 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.749791 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.750029 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.750044 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.750045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.750177 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.750499 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:57.750506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:57.750670 (+ 164us) client_negotiation.cc:769] Sending connection context
0925 14:44:57.750778 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 14:44:57.750919 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
I20260925 14:44:57.751645 24284 heartbeater.cc:347] Connected to a master server at 127.21.205.254:38521
I20260925 14:44:57.751729 24284 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:57.751950 24284 heartbeater.cc:511] Master 127.21.205.254:38521 requested a full tablet report, sending...
I20260925 14:44:57.752512 24078 ts_manager.cc:194] Registered new tserver with Master: a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425)
I20260925 14:44:57.753319 24078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:44819
I20260925 14:44:57.755522 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24151
I20260925 14:44:57.755616 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:44:57.755813 22327 external_mini_cluster.cc:1428] Setting key afdd04e25408e444d3378d7a1a76869e
I20260925 14:44:57.757853 24290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.752668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.193:56351)
0925 14:44:57.752812 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:57.752821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:57.752876 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:57.753000 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:57.753000 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:57.753130 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:57.753185 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.753186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.753970 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.753971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.754456 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.754458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.754782 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.754782 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.754834 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.755135 (+ 301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:57.755138 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:57.755401 (+ 263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:57.757024 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.757024 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.757025 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.757174 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.757355 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.757356 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.757356 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.757401 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.757629 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:57.757630 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:57.757680 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 14:44:57.757728 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 14:44:57.757794 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260925 14:44:57.757983 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.752717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56351 (local address 127.21.205.254:38521)
0925 14:44:57.752761 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:57.752763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:57.752913 (+ 150us) server_negotiation.cc:407] Connection header received
0925 14:44:57.752935 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:57.752936 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:57.752945 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:57.752997 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:57.753243 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.753878 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.754544 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.754713 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.754770 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.755492 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:57.755493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:57.755494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:57.755500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:57.756947 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.757237 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.757238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.757238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.757294 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.757451 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.757452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.757453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.757575 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:57.757634 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:57.757776 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 14:44:57.757854 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:44:57.778816 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:38521
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:57.858403 24293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:57.858582 24293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:57.858605 24293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:57.859630 24293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:57.859661 24293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:57.859679 24293 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 14:44:57.859691 24293 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:57.859707 24293 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:44:57.861035 24293 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:38521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:57.861202 24293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:57.861368 24293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:57.866113 24301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:57.866187 24298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:57.866115 24299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:57.866245 24293 server_base.cc:1034] running on GCE node
I20260925 14:44:57.866537 24293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:57.866784 24293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:57.867945 24293 hybrid_clock.cc:648] HybridClock initialized: now 1790347497867944 us; error 38 us; skew 500 ppm
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:57.876732 24293 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:44:57.877151 24293 webserver.cc:460] Webserver started at http://127.21.205.194:38203/ using document root <none> and password file <none>
I20260925 14:44:57.877384 24293 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:57.877450 24293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:57.877601 24293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:57.878698 24293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "8d75935ed62e4e158ba4102541b16ec0"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "c845acfe11651ffad75167f8ae655217"
server_key_iv: "d802b88da4a47806f0701dbacf865a8b"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.879062 24293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "8d75935ed62e4e158ba4102541b16ec0"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "c845acfe11651ffad75167f8ae655217"
server_key_iv: "d802b88da4a47806f0701dbacf865a8b"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.880434 24293 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:44:57.881143 24309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:57.881304 24293 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:57.881402 24293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "8d75935ed62e4e158ba4102541b16ec0"
format_stamp: "Formatted at 2026-09-25 14:44:57 on dist-test-slave-d7lw"
server_key: "c845acfe11651ffad75167f8ae655217"
server_key_iv: "d802b88da4a47806f0701dbacf865a8b"
server_key_version: "encryptionkey@0"
I20260925 14:44:57.881479 24293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:57.896596 24293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:57.898692 24293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:57.898841 24293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:57.899314 24293 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:57.899387 24293 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:57.899453 24293 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:57.899477 24293 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:57 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:57.906296 24293 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:42947
I20260925 14:44:57.906348 24424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:42947 every 8 connection(s)
I20260925 14:44:57.906678 24293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:44:57.909106 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.899854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:39317 (local address 127.21.205.254:38521)
0925 14:44:57.899929 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:57.899931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:57.900138 (+ 207us) server_negotiation.cc:407] Connection header received
0925 14:44:57.900583 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:57.900584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:57.900597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:57.900682 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:57.901252 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.902110 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.903682 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.904056 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.904138 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.906626 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:57.906627 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:57.906628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:57.906636 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:57.908055 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.908401 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.908402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.908403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.908453 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.908649 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.908649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.908649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.908742 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:57.908791 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:57.908990 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 14:44:57.909046 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:44:57.909353 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.899853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.194:39317)
0925 14:44:57.900015 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:57.900052 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:57.900496 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:57.900748 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:57.900751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:57.900954 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:57.901178 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.901182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.902204 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.902206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.903588 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.903591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.904156 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.904158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.904315 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.904833 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:57.904840 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:57.906529 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:57.908121 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.908126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.908127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.908338 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.908508 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.908508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.908508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.908596 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.908795 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:57.908799 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:57.908899 (+ 100us) client_negotiation.cc:769] Sending connection context
0925 14:44:57.908975 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:44:57.909102 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":98,"threads_started":1}
I20260925 14:44:57.909834 24425 heartbeater.cc:347] Connected to a master server at 127.21.205.254:38521
I20260925 14:44:57.909914 24425 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:57.910077 24425 heartbeater.cc:511] Master 127.21.205.254:38521 requested a full tablet report, sending...
I20260925 14:44:57.910415 24079 ts_manager.cc:194] Registered new tserver with Master: 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:44:57.910928 24079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:39317
I20260925 14:44:57.912864 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24293
I20260925 14:44:57.912974 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:44:57.913132 22327 external_mini_cluster.cc:1428] Setting key e26f86d43b4f35d0fd7b4dd2844f783d
I20260925 14:44:57.916714 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.910802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:42123 (local address 127.21.205.254:38521)
0925 14:44:57.910837 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:57.910839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:57.910902 (+ 63us) server_negotiation.cc:407] Connection header received
0925 14:44:57.910964 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:57.910965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:57.910971 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:57.911040 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:57.911333 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.911760 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.912474 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.912662 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.912740 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.913710 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:57.913711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:57.913711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:57.913719 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:57.915371 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.915818 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.915819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.915820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.915881 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:57.916129 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:57.916130 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:57.916130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:57.916282 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:57.916375 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:57.916550 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 14:44:57.916625 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:44:57.916734 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:57.910709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.194:42123)
0925 14:44:57.910836 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:57.910845 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:57.910913 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:57.911057 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:57.911058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:57.911200 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:57.911250 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.911251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.911870 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.911871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.912393 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:57.912394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:57.912747 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:57.912748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:57.912829 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:57.913212 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:57.913217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:57.913610 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:57.915482 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.915483 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.915483 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.915707 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.915964 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:57.915965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:57.915965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:57.916032 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:57.916383 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:57.916384 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:57.916430 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 14:44:57.916522 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 14:44:57.916625 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:44:57.938108 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:38521
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:44:58.018000 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:44:58.018306 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:44:58.018349 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:44:58.019491 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:44:58.019531 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:44:58.019550 24434 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 14:44:58.019562 24434 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:44:58.019603 24434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:44:58.021159 24434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:38521
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:44:58.021397 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:44:58.021606 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:44:58.026345 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:58.026441 24434 server_base.cc:1034] running on GCE node
W20260925 14:44:58.026407 24443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:44:58.026343 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:44:58.026800 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:44:58.027084 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:44:58.028239 24434 hybrid_clock.cc:648] HybridClock initialized: now 1790347498028233 us; error 38 us; skew 500 ppm
Sep 25 14:44:58 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347498, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:44:58.030403 24434 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:44:58.030817 24434 webserver.cc:460] Webserver started at http://127.21.205.195:33209/ using document root <none> and password file <none>
I20260925 14:44:58.031030 24434 fs_manager.cc:359] Metadata directory not provided
I20260925 14:44:58.031092 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:44:58.031241 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:44:58.032346 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "75b350545a904dc2bae94677003887d0"
format_stamp: "Formatted at 2026-09-25 14:44:58 on dist-test-slave-d7lw"
server_key: "5c5599b7f1b4a868dd37119a4d6a1d08"
server_key_iv: "53bec0b107d47127d2c24edd790b68f8"
server_key_version: "encryptionkey@0"
I20260925 14:44:58.032681 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "75b350545a904dc2bae94677003887d0"
format_stamp: "Formatted at 2026-09-25 14:44:58 on dist-test-slave-d7lw"
server_key: "5c5599b7f1b4a868dd37119a4d6a1d08"
server_key_iv: "53bec0b107d47127d2c24edd790b68f8"
server_key_version: "encryptionkey@0"
I20260925 14:44:58.033991 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:44:58.034695 24450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.034859 24434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:44:58.034951 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "75b350545a904dc2bae94677003887d0"
format_stamp: "Formatted at 2026-09-25 14:44:58 on dist-test-slave-d7lw"
server_key: "5c5599b7f1b4a868dd37119a4d6a1d08"
server_key_iv: "53bec0b107d47127d2c24edd790b68f8"
server_key_version: "encryptionkey@0"
I20260925 14:44:58.035022 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:44:58.049626 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:44:58.051578 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:44:58.051728 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:44:58.052316 24434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:44:58.052364 24434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.052392 24434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:44:58.052415 24434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:44:58 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347498, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:58.059369 24434 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:40993
I20260925 14:44:58.059755 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:44:58.059801 24565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:40993 every 8 connection(s)
I20260925 14:44:58.061507 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.052677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:55203 (local address 127.21.205.254:38521)
0925 14:44:58.052717 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.052719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.052905 (+ 186us) server_negotiation.cc:407] Connection header received
0925 14:44:58.053405 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.053407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.053418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.053473 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:58.054269 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.055116 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.056365 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.056710 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.056831 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.058895 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:58.058896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:58.058897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:58.058904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:58.060495 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.060818 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.060819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.060819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.060867 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.061060 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.061060 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.061060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.061160 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:58.061201 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.061392 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.061449 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:44:58.061709 24459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.052648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.195:55203)
0925 14:44:58.052797 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.052828 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.053277 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.053533 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.053538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.053879 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:58.054157 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.054163 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.055202 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.055203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.056257 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.056260 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.056864 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.056865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.056999 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.057488 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:58.057496 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:58.058804 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:58.060560 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.060564 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.060565 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.060767 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.060921 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.060922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.060922 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.061004 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.061208 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:58.061211 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:58.061305 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.061379 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.061479 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260925 14:44:58.062193 24566 heartbeater.cc:347] Connected to a master server at 127.21.205.254:38521
I20260925 14:44:58.062275 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24434
I20260925 14:44:58.062300 24566 heartbeater.cc:464] Registering TS with master...
I20260925 14:44:58.062378 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:44:58.062470 24566 heartbeater.cc:511] Master 127.21.205.254:38521 requested a full tablet report, sending...
I20260925 14:44:58.062551 22327 external_mini_cluster.cc:1428] Setting key 767fb39ddb9e8242f71d3bb067403722
I20260925 14:44:58.063336 24078 ts_manager.cc:194] Registered new tserver with Master: 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993)
I20260925 14:44:58.063655 24078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:55203
I20260925 14:44:58.068720 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.063252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:47347 (local address 127.21.205.254:38521)
0925 14:44:58.063285 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.063298 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.063883 (+ 585us) server_negotiation.cc:407] Connection header received
0925 14:44:58.063949 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.063950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.063958 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.063998 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:58.064360 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.064789 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.065443 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.065637 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.065703 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.066522 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:58.066523 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:58.066524 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:58.066530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:58.067678 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.068010 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.068011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.068012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.068061 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.068256 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.068257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.068257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.068356 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:58.068425 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.068567 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.068641 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:44:58.068735 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.063188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.21.205.195:47347)
0925 14:44:58.063834 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.063844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.063895 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.064012 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.064013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.064204 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:58.064261 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.064262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.064879 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.064879 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.065362 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.065364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.065715 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.065715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.065787 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.066172 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:58.066176 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:58.066420 (+ 244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:58.067757 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.067758 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.067758 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.067940 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.068133 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.068134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.068134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.068187 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.068418 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:58.068419 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:58.068459 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.068543 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.068636 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":79,"threads_started":1}
I20260925 14:44:58.074066 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 14:44:58 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 14:44:58 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:58.095729 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.089193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.21.205.254:38521)
0925 14:44:58.089251 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.089253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.089336 (+ 83us) server_negotiation.cc:407] Connection header received
0925 14:44:58.089407 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.089408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.089418 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.089465 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:58.089986 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.090425 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.091102 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.091292 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.091369 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.093596 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:58.093597 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:58.093598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:58.093605 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:58.094692 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.095014 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.095015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.095015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.095069 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.095257 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.095258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.095258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.095355 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:58.095451 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.095566 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.095640 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:44:58.097127 24078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:44:58.098709 24078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:58.105031 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.101073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.21.205.195:40993)
0925 14:44:58.101317 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.101321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.101518 (+ 197us) server_negotiation.cc:407] Connection header received
0925 14:44:58.101571 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.101572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.101583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.101673 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.101830 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.102672 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.103990 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.104662 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.104744 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.104796 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.104919 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.104953 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":86,"threads_started":1}
I20260925 14:44:58.105023 24584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.100993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40993 (local address 127.0.0.1:49190)
0925 14:44:58.101320 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.101354 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.101421 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.101677 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.101679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.101682 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.101737 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.101738 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.102756 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.102757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.103911 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.103914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.104745 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.104746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.104823 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.104835 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.104902 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.104950 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":51,"threads_started":1}
I20260925 14:44:58.105295 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.100993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.0.0.1:32920)
0925 14:44:58.101349 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.101359 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.101422 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.101786 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.101787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.101789 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.101854 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.101855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.103163 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.103164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.104178 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.104180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.105050 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.105051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.105111 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.105115 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.105175 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.105214 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":2,"thread_start_us":66,"threads_started":1}
I20260925 14:44:58.105513 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.101179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42947 (local address 127.0.0.1:57772)
0925 14:44:58.101778 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.101787 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.101851 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.102101 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.102102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.102104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.102159 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.102161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.103357 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.103358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.104442 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.104444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.105289 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.105290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.105350 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.105354 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.105401 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.105446 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260925 14:44:58.105548 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.101173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.21.205.193:46425)
0925 14:44:58.101512 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.101517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.101527 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:44:58.101573 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.101575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.101614 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.101687 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.101948 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.103074 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.104267 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.104900 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.104970 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.105015 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.105401 (+ 386us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.105445 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":73,"threads_started":1}
I20260925 14:44:58.106001 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.101285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57772 (local address 127.21.205.194:42947)
0925 14:44:58.101433 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.101437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.101923 (+ 486us) server_negotiation.cc:407] Connection header received
0925 14:44:58.101984 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.101986 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.102002 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.102087 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.102250 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.103262 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.104528 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.105215 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.105278 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.105323 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.105873 (+ 550us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.105916 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260925 14:44:58.106091 24500 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7dff2935e14b3599a8059802b301c7 (DEFAULT_TABLE table=test-table [id=f7f36dfdd46844be8259c009b8df85a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.106220 24218 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7dff2935e14b3599a8059802b301c7 (DEFAULT_TABLE table=test-table [id=f7f36dfdd46844be8259c009b8df85a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.106426 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7dff2935e14b3599a8059802b301c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.106493 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7dff2935e14b3599a8059802b301c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.106544 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7dff2935e14b3599a8059802b301c7 (DEFAULT_TABLE table=test-table [id=f7f36dfdd46844be8259c009b8df85a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.106827 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7dff2935e14b3599a8059802b301c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.109129 24590 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Bootstrap starting.
I20260925 14:44:58.109889 24590 tablet_bootstrap.cc:654] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.110183 24590 log.cc:824] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:58.110455 24592 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Bootstrap starting.
I20260925 14:44:58.110628 24591 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Bootstrap starting.
I20260925 14:44:58.110996 24590 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: No bootstrap required, opened a new log
I20260925 14:44:58.111085 24590 ts_tablet_manager.cc:1397] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:44:58.111208 24592 tablet_bootstrap.cc:654] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.111526 24591 tablet_bootstrap.cc:654] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.111552 24592 log.cc:824] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:58.111894 24591 log.cc:824] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:44:58.112494 24592 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: No bootstrap required, opened a new log
I20260925 14:44:58.112597 24592 ts_tablet_manager.cc:1397] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:44:58.112614 24591 tablet_bootstrap.cc:492] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: No bootstrap required, opened a new log
I20260925 14:44:58.112677 24591 ts_tablet_manager.cc:1397] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 14:44:58.113267 24590 raft_consensus.cc:348] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.113442 24590 raft_consensus.cc:374] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.113515 24590 raft_consensus.cc:729] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.113639 24590 consensus_queue.cc:260] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.113925 24590 ts_tablet_manager.cc:1428] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:44:58.113965 24284 heartbeater.cc:503] Master 127.21.205.254:38521 was elected leader, sending a full tablet report...
I20260925 14:44:58.114571 24591 raft_consensus.cc:348] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.114634 24592 raft_consensus.cc:348] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.114738 24591 raft_consensus.cc:374] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.114761 24592 raft_consensus.cc:374] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.114785 24591 raft_consensus.cc:729] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b350545a904dc2bae94677003887d0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.114835 24592 raft_consensus.cc:729] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.114880 24591 consensus_queue.cc:260] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.114984 24592 consensus_queue.cc:260] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.115152 24591 ts_tablet_manager.cc:1428] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 14:44:58.115245 24566 heartbeater.cc:503] Master 127.21.205.254:38521 was elected leader, sending a full tablet report...
I20260925 14:44:58.115566 24592 ts_tablet_manager.cc:1428] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 14:44:58.115653 24425 heartbeater.cc:503] Master 127.21.205.254:38521 was elected leader, sending a full tablet report...
I20260925 14:44:58.144259 24598 raft_consensus.cc:482] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:58.144340 24598 raft_consensus.cc:504] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.144678 24598 leader_election.cc:288] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993), a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425)
I20260925 14:44:58.148206 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.144906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:55513 (local address 127.21.205.193:46425)
0925 14:44:58.144985 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.144987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.144995 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:44:58.145019 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.145020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.145029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.145083 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.145273 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.145997 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147271 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.147842 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147899 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.147905 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.148112 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.148145 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 14:44:58.148206 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.144919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:37141 (local address 127.21.205.195:40993)
0925 14:44:58.144974 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.144976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.145260 (+ 284us) server_negotiation.cc:407] Connection header received
0925 14:44:58.145286 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.145287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.145295 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.145347 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.145445 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.146124 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147289 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.147825 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147897 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.147904 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.148108 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.148145 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:44:58.148242 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.144845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.21.205.194:55513)
0925 14:44:58.144885 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.144895 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.144968 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.145114 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.145116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.145128 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.145196 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.145198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.146064 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.146064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.147168 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.147170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.147908 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147921 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.148025 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.148027 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.148077 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.148143 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":6}
I20260925 14:44:58.148259 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.144845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40993 (local address 127.21.205.194:37141)
0925 14:44:58.145165 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.145176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.145238 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.145358 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.145359 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.145360 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.145403 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.145404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.146186 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.146186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.147218 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.147220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.147908 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.147921 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.148014 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.148018 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.148077 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.148145 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":201,"thread_start_us":78,"threads_started":1}
I20260925 14:44:58.148645 24516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7dff2935e14b3599a8059802b301c7" candidate_uuid: "8d75935ed62e4e158ba4102541b16ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b350545a904dc2bae94677003887d0" is_pre_election: true
I20260925 14:44:58.148665 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7dff2935e14b3599a8059802b301c7" candidate_uuid: "8d75935ed62e4e158ba4102541b16ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea" is_pre_election: true
I20260925 14:44:58.148777 24228 raft_consensus.cc:2454] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d75935ed62e4e158ba4102541b16ec0 in term 0.
I20260925 14:44:58.148777 24516 raft_consensus.cc:2454] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d75935ed62e4e158ba4102541b16ec0 in term 0.
I20260925 14:44:58.148998 24310 leader_election.cc:302] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d75935ed62e4e158ba4102541b16ec0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:44:58.149130 24598 raft_consensus.cc:2781] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:58.149217 24598 raft_consensus.cc:482] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:58.149240 24598 raft_consensus.cc:3037] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.149859 24598 raft_consensus.cc:504] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.149992 24598 leader_election.cc:288] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [CANDIDATE]: Term 1 election: Requested vote from peers 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993), a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425)
I20260925 14:44:58.150132 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7dff2935e14b3599a8059802b301c7" candidate_uuid: "8d75935ed62e4e158ba4102541b16ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea"
I20260925 14:44:58.150130 24516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7dff2935e14b3599a8059802b301c7" candidate_uuid: "8d75935ed62e4e158ba4102541b16ec0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b350545a904dc2bae94677003887d0"
I20260925 14:44:58.150199 24516 raft_consensus.cc:3037] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.150233 24228 raft_consensus.cc:3037] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.150807 24516 raft_consensus.cc:2454] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d75935ed62e4e158ba4102541b16ec0 in term 1.
I20260925 14:44:58.150951 24228 raft_consensus.cc:2454] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d75935ed62e4e158ba4102541b16ec0 in term 1.
I20260925 14:44:58.151295 24313 leader_election.cc:302] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, 8d75935ed62e4e158ba4102541b16ec0; no voters:
I20260925 14:44:58.151427 24598 raft_consensus.cc:2781] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:58.151572 24598 raft_consensus.cc:686] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 LEADER]: Becoming Leader. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Running, Role: LEADER
I20260925 14:44:58.151675 24598 consensus_queue.cc:237] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } }
I20260925 14:44:58.152426 24079 catalog_manager.cc:5495] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "8d75935ed62e4e158ba4102541b16ec0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } health_report { overall_health: UNKNOWN } } }
W20260925 14:44:58.157233 24426 tablet.cc:2369] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 14:44:58 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347497, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:44:58.186378 24128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.180368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.21.205.254:38521)
0925 14:44:58.180417 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.180418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.180536 (+ 118us) server_negotiation.cc:407] Connection header received
0925 14:44:58.180567 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.180568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.180576 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.180614 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:44:58.180998 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.181402 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.181985 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.182139 (+ 154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.182204 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.184121 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:44:58.184122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:44:58.184122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:44:58.184128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:44:58.185475 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.185823 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.185823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.185824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.185869 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:44:58.186022 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:44:58.186022 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:44:58.186022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:44:58.186102 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:44:58.186145 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.186260 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.186320 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 14:44:58.186534 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.180296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:38521 (local address 127.0.0.1:50574)
0925 14:44:58.180465 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.180472 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.180525 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.180643 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.180644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.180903 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:44:58.180938 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.180940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.181460 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.181460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.181916 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.181917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.182252 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.182253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.182302 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.182746 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:44:58.182751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:44:58.184053 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:44:58.185561 (+ 1508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.185567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.185569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.185758 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.185925 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:44:58.185925 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:44:58.185926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:44:58.185977 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:44:58.186153 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:44:58.186158 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:44:58.186195 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.186247 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.186370 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":40,"thread_start_us":90,"threads_started":1}
I20260925 14:44:58.188211 24079 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:50574:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 14:44:58.188395 24079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:58.189772 24218 tablet_service.cc:1467] Processing CreateTablet for tablet 21dd87ab39f949a0a0041a2560b4dae6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c1e6e7ec029401c96a8f917211bac1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:58.189759 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 21dd87ab39f949a0a0041a2560b4dae6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c1e6e7ec029401c96a8f917211bac1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:58.189783 24500 tablet_service.cc:1467] Processing CreateTablet for tablet 21dd87ab39f949a0a0041a2560b4dae6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4c1e6e7ec029401c96a8f917211bac1a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:44:58.189945 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21dd87ab39f949a0a0041a2560b4dae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.189945 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21dd87ab39f949a0a0041a2560b4dae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.189946 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21dd87ab39f949a0a0041a2560b4dae6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.190974 24590 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: Bootstrap starting.
I20260925 14:44:58.191299 24591 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: Bootstrap starting.
I20260925 14:44:58.191696 24590 tablet_bootstrap.cc:654] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.191871 24591 tablet_bootstrap.cc:654] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.192255 24592 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: Bootstrap starting.
I20260925 14:44:58.192363 24590 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: No bootstrap required, opened a new log
I20260925 14:44:58.192415 24590 ts_tablet_manager.cc:1397] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 14:44:58.192577 24590 raft_consensus.cc:348] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.192662 24592 tablet_bootstrap.cc:654] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.192682 24590 raft_consensus.cc:374] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.192708 24590 raft_consensus.cc:729] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.192763 24590 consensus_queue.cc:260] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.192867 24590 ts_tablet_manager.cc:1428] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.193017 24596 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.193112 24596 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.193259 24591 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: No bootstrap required, opened a new log
I20260925 14:44:58.193313 24591 ts_tablet_manager.cc:1397] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 14:44:58.193516 24592 tablet_bootstrap.cc:492] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: No bootstrap required, opened a new log
I20260925 14:44:58.193482 24591 raft_consensus.cc:348] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.193547 24591 raft_consensus.cc:374] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.193557 24592 ts_tablet_manager.cc:1397] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:44:58.193571 24591 raft_consensus.cc:729] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b350545a904dc2bae94677003887d0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.193629 24591 consensus_queue.cc:260] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.193729 24591 ts_tablet_manager.cc:1428] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.193688 24592 raft_consensus.cc:348] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.193771 24592 raft_consensus.cc:374] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.193797 24592 raft_consensus.cc:729] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.193787 24597 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.193845 24592 consensus_queue.cc:260] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.193897 24597 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.193930 24592 ts_tablet_manager.cc:1428] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.193956 24598 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.194023 24598 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260925 14:44:58.249383 24285 tablet.cc:2369] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:44:58.310379 24567 tablet.cc:2369] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:44:58.447736 24596 raft_consensus.cc:482] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:58.447942 24596 raft_consensus.cc:504] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.448323 24596 leader_election.cc:288] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947), 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993)
I20260925 14:44:58.451326 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.448817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:51877 (local address 127.21.205.195:40993)
0925 14:44:58.448884 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.448886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.448896 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:44:58.448967 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.448968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.448977 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.449028 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.449191 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.449649 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.450592 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.450982 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.451064 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.451073 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.451231 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.451268 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:44:58.451328 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.448707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40993 (local address 127.21.205.193:51877)
0925 14:44:58.448814 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.448823 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.448897 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.449044 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.449045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.449047 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.449118 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.449119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.449736 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.449737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.450501 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.450503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.451064 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.451065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.451141 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.451145 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.451207 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.451249 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":5,"thread_start_us":35,"threads_started":1}
I20260925 14:44:58.451437 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.448587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:53327 (local address 127.21.205.194:42947)
0925 14:44:58.448663 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.448666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.448855 (+ 189us) server_negotiation.cc:407] Connection header received
0925 14:44:58.448889 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.448890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.448900 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.448986 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.449170 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.449672 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.450666 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.451080 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.451166 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.451174 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.451331 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.451369 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:44:58.451452 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.448513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42947 (local address 127.21.205.193:53327)
0925 14:44:58.448673 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.448686 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.448806 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.449005 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.449006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.449009 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.449095 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.449097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.449767 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.449768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.450523 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.450525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.451167 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.451168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.451248 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.451252 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.451325 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.451369 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260925 14:44:58.451658 24516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21dd87ab39f949a0a0041a2560b4dae6" candidate_uuid: "a0825ee4b4394c0d84229620848b07ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b350545a904dc2bae94677003887d0" is_pre_election: true
I20260925 14:44:58.451756 24516 raft_consensus.cc:2454] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0825ee4b4394c0d84229620848b07ea in term 0.
I20260925 14:44:58.451828 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21dd87ab39f949a0a0041a2560b4dae6" candidate_uuid: "a0825ee4b4394c0d84229620848b07ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0" is_pre_election: true
I20260925 14:44:58.451975 24379 raft_consensus.cc:2454] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0825ee4b4394c0d84229620848b07ea in term 0.
I20260925 14:44:58.451972 24170 leader_election.cc:302] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:44:58.452095 24596 raft_consensus.cc:2781] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:58.452157 24596 raft_consensus.cc:482] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:58.452178 24596 raft_consensus.cc:3037] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.452929 24596 raft_consensus.cc:504] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.453070 24596 leader_election.cc:288] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [CANDIDATE]: Term 1 election: Requested vote from peers 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947), 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993)
I20260925 14:44:58.453270 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21dd87ab39f949a0a0041a2560b4dae6" candidate_uuid: "a0825ee4b4394c0d84229620848b07ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0"
I20260925 14:44:58.453294 24516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21dd87ab39f949a0a0041a2560b4dae6" candidate_uuid: "a0825ee4b4394c0d84229620848b07ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75b350545a904dc2bae94677003887d0"
I20260925 14:44:58.453364 24379 raft_consensus.cc:3037] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.453364 24516 raft_consensus.cc:3037] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.454116 24516 raft_consensus.cc:2454] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0825ee4b4394c0d84229620848b07ea in term 1.
I20260925 14:44:58.454116 24379 raft_consensus.cc:2454] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0825ee4b4394c0d84229620848b07ea in term 1.
I20260925 14:44:58.454305 24170 leader_election.cc:302] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d75935ed62e4e158ba4102541b16ec0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:44:58.454396 24596 raft_consensus.cc:2781] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:58.454519 24596 raft_consensus.cc:686] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [term 1 LEADER]: Becoming Leader. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Running, Role: LEADER
I20260925 14:44:58.454661 24596 consensus_queue.cc:237] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.455107 24613 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: TxnStatusTablet state changed. Reason: New leader a0825ee4b4394c0d84229620848b07ea. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.455232 24613 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:58.455416 24615 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 14:44:58.455554 24615 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 14:44:58.455525 24079 catalog_manager.cc:5495] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea reported cstate change: term changed from 0 to 1, leader changed from <none> to a0825ee4b4394c0d84229620848b07ea (127.21.205.193). New cstate: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } health_report { overall_health: UNKNOWN } } }
I20260925 14:44:58.475958 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.473628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32928 (local address 127.21.205.193:46425)
0925 14:44:58.473698 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.473700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.473709 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:58.473754 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.473755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.473779 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.473849 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.473975 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.474440 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.475179 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.475366 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.475438 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.475613 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.475708 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.475812 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.475868 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.475890 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 14:44:58.475977 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.473527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.0.0.1:32928)
0925 14:44:58.473598 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.473609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.473671 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.473856 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.473857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.473858 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:58.473908 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.473910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.474532 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.474533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.475088 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.475090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.475437 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.475438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.475503 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.475524 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.475777 (+ 253us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.475809 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.475863 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.475909 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 14:44:58.477612 24516 raft_consensus.cc:1261] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Refusing update from remote peer a0825ee4b4394c0d84229620848b07ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.477644 24379 raft_consensus.cc:1261] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Refusing update from remote peer a0825ee4b4394c0d84229620848b07ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.477929 24596 consensus_queue.cc:1035] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.478101 24613 consensus_queue.cc:1035] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.479278 24597 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: TxnStatusTablet state changed. Reason: New leader a0825ee4b4394c0d84229620848b07ea. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.479387 24597 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.479446 24598 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: TxnStatusTablet state changed. Reason: New leader a0825ee4b4394c0d84229620848b07ea. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.479525 24598 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.480551 24596 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.480638 24596 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:58.480906 24613 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.480981 24613 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P a0825ee4b4394c0d84229620848b07ea: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:44:58.481053 24598 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.481120 24598 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 8d75935ed62e4e158ba4102541b16ec0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.481246 24620 mvcc.cc:205] Tried to move back new op lower bound from 7333263353761554432 to 7333263353670873088. Current Snapshot: MvccSnapshot[applied={T|T < 7333263353761554432}]
I20260925 14:44:58.481986 24597 tablet_replica.cc:439] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a0825ee4b4394c0d84229620848b07ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } } }
I20260925 14:44:58.482087 24597 tablet_replica.cc:442] T 21dd87ab39f949a0a0041a2560b4dae6 P 75b350545a904dc2bae94677003887d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:44:58.482586 24618 mvcc.cc:205] Tried to move back new op lower bound from 7333263353761554432 to 7333263353670873088. Current Snapshot: MvccSnapshot[applied={T|T < 7333263353761554432 or (T in {7333263353761554432})}]
I20260925 14:44:58.485957 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.483486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57786 (local address 127.21.205.194:42947)
0925 14:44:58.483528 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.483530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.483539 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:58.483599 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.483600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.483615 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.483685 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.483809 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.484319 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.485121 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.485380 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.485468 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.485629 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.485752 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.485835 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.485871 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.485888 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:44:58.491017 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.488451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:56937 (local address 127.21.205.193:46425)
0925 14:44:58.488492 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.488494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.488568 (+ 74us) server_negotiation.cc:407] Connection header received
0925 14:44:58.488678 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.488679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.488687 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.488740 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.488886 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.489359 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.490383 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.490548 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.490590 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.490808 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.490831 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.490870 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.490959 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.490975 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:44:58.491117 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.488366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.21.205.194:56937)
0925 14:44:58.488510 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.488521 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.488589 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.488759 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.488759 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.488761 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:58.488819 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.488821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.489428 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.489429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.490293 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.490296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.490627 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.490629 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.490696 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.490711 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.490893 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.490895 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.490957 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.490998 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260925 14:44:58.493698 24516 raft_consensus.cc:1261] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Refusing update from remote peer 8d75935ed62e4e158ba4102541b16ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.493732 24228 raft_consensus.cc:1261] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Refusing update from remote peer 8d75935ed62e4e158ba4102541b16ec0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.493976 24598 consensus_queue.cc:1035] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.494105 24600 consensus_queue.cc:1035] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.501832 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.499296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56829 (local address 127.21.205.194:42947)
0925 14:44:58.499334 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.499336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.499518 (+ 182us) server_negotiation.cc:407] Connection header received
0925 14:44:58.499539 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.499540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.499549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.499606 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.499729 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.500214 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.501258 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.501437 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.501485 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.501665 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.501687 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.501721 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.501780 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.501794 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:44:58.501951 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.499258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42947 (local address 127.21.205.193:56829)
0925 14:44:58.499432 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.499439 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.499482 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.499609 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.499609 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.499610 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:58.499647 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.499648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.500304 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.500305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.501177 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.501179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.501508 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.501509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.501576 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.501591 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.501725 (+ 134us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.501726 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.501780 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.501819 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":73,"threads_started":1}
I20260925 14:44:58.510986 24079 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:58.511092 24079 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:44:58.511864 24079 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Sending DeleteTablet for 3 replicas of tablet 6d7dff2935e14b3599a8059802b301c7
I20260925 14:44:58.512213 24500 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7dff2935e14b3599a8059802b301c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260925 14:44:58.512215 24359 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7dff2935e14b3599a8059802b301c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:57772
I20260925 14:44:58.512352 24079 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 14:44:58.512406 24218 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7dff2935e14b3599a8059802b301c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:44:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32920
W20260925 14:44:58.512491 24079 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:44:58.512635 24640 tablet_replica.cc:330] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: stopping tablet replica
I20260925 14:44:58.512733 24640 raft_consensus.cc:2229] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:58.512789 24640 raft_consensus.cc:2258] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:58.513206 24640 ts_tablet_manager.cc:1905] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:58.513772 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (DEFAULT_TABLE table=test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.513816 24500 tablet_service.cc:1467] Processing CreateTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (DEFAULT_TABLE table=test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.513882 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6363cb4bf1ec4e359b750ab09cbc3e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.513901 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6363cb4bf1ec4e359b750ab09cbc3e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.515228 24640 ts_tablet_manager.cc:1918] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:58.515285 24592 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Bootstrap starting.
I20260925 14:44:58.515316 24640 log.cc:1196] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/6d7dff2935e14b3599a8059802b301c7
I20260925 14:44:58.515446 24218 tablet_service.cc:1467] Processing CreateTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (DEFAULT_TABLE table=test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:44:58.515539 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6363cb4bf1ec4e359b750ab09cbc3e80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:44:58.515645 24640 ts_tablet_manager.cc:1939] T 6d7dff2935e14b3599a8059802b301c7 P a0825ee4b4394c0d84229620848b07ea: Deleting consensus metadata
I20260925 14:44:58.515751 24592 tablet_bootstrap.cc:654] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.516794 24591 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Bootstrap starting.
I20260925 14:44:58.516798 24590 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Bootstrap starting.
I20260925 14:44:58.517164 24591 tablet_bootstrap.cc:654] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.517210 24590 tablet_bootstrap.cc:654] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:44:58.517611 24063 catalog_manager.cc:4841] TS a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425): tablet 6d7dff2935e14b3599a8059802b301c7 (table test-table [id=f7f36dfdd46844be8259c009b8df85a8]) successfully deleted
I20260925 14:44:58.517777 24590 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: No bootstrap required, opened a new log
I20260925 14:44:58.517827 24590 ts_tablet_manager.cc:1397] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260925 14:44:58.517974 24590 raft_consensus.cc:348] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.518031 24590 raft_consensus.cc:374] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.518052 24590 raft_consensus.cc:729] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.518106 24590 consensus_queue.cc:260] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.518203 24590 ts_tablet_manager.cc:1428] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.520118 24639 tablet_replica.cc:330] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: stopping tablet replica
I20260925 14:44:58.520169 24592 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: No bootstrap required, opened a new log
I20260925 14:44:58.520208 24592 ts_tablet_manager.cc:1397] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 14:44:58.520227 24639 raft_consensus.cc:2229] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:44:58.520291 24639 raft_consensus.cc:2258] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:58.520385 24592 raft_consensus.cc:348] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.520452 24592 raft_consensus.cc:374] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.520468 24592 raft_consensus.cc:729] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.520524 24592 consensus_queue.cc:260] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.520637 24592 ts_tablet_manager.cc:1428] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.520939 24638 tablet_replica.cc:330] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: stopping tablet replica
I20260925 14:44:58.521046 24638 raft_consensus.cc:2229] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:44:58.521209 24638 raft_consensus.cc:2258] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:44:58.521212 24639 ts_tablet_manager.cc:1905] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:58.521723 24591 tablet_bootstrap.cc:492] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: No bootstrap required, opened a new log
I20260925 14:44:58.521775 24591 ts_tablet_manager.cc:1397] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260925 14:44:58.521915 24591 raft_consensus.cc:348] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.521978 24591 raft_consensus.cc:374] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:44:58.521998 24591 raft_consensus.cc:729] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b350545a904dc2bae94677003887d0, State: Initialized, Role: FOLLOWER
I20260925 14:44:58.522042 24591 consensus_queue.cc:260] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.522130 24591 ts_tablet_manager.cc:1428] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:44:58.523651 24639 ts_tablet_manager.cc:1918] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:58.523777 24639 log.cc:1196] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/6d7dff2935e14b3599a8059802b301c7
I20260925 14:44:58.524149 24639 ts_tablet_manager.cc:1939] T 6d7dff2935e14b3599a8059802b301c7 P 75b350545a904dc2bae94677003887d0: Deleting consensus metadata
I20260925 14:44:58.524545 24066 catalog_manager.cc:4841] TS 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993): tablet 6d7dff2935e14b3599a8059802b301c7 (table test-table [id=f7f36dfdd46844be8259c009b8df85a8]) successfully deleted
I20260925 14:44:58.528024 24638 ts_tablet_manager.cc:1905] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:44:58.529560 24638 ts_tablet_manager.cc:1918] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:44:58.529633 24638 log.cc:1196] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/6d7dff2935e14b3599a8059802b301c7
I20260925 14:44:58.529929 24638 ts_tablet_manager.cc:1939] T 6d7dff2935e14b3599a8059802b301c7 P 8d75935ed62e4e158ba4102541b16ec0: Deleting consensus metadata
I20260925 14:44:58.530313 24066 catalog_manager.cc:4841] TS 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947): tablet 6d7dff2935e14b3599a8059802b301c7 (table test-table [id=f7f36dfdd46844be8259c009b8df85a8]) successfully deleted
I20260925 14:44:58.773010 24597 raft_consensus.cc:482] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:44:58.773104 24597 raft_consensus.cc:504] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.773432 24597 leader_election.cc:288] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:44:58.777424 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.773686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:52219 (local address 127.21.205.194:42947)
0925 14:44:58.773738 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.773740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.773839 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:44:58.773896 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.773897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.773911 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.774013 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.774217 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.774949 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.776502 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.777046 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.777122 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.777131 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.777321 (+ 190us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.777358 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:44:58.777424 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.773698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:42355 (local address 127.21.205.193:46425)
0925 14:44:58.773768 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.773770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.773834 (+ 64us) server_negotiation.cc:407] Connection header received
0925 14:44:58.773900 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.773901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.773911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.773966 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:44:58.774196 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.774949 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.776502 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.777055 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.777123 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.777131 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.777320 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.777355 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 14:44:58.777469 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.773607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42947 (local address 127.21.205.195:52219)
0925 14:44:58.773766 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.773777 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.773832 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.774002 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.774003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.774004 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.774109 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.774112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.775035 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.775036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.776411 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.776414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.777133 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.777134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.777233 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.777240 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.777301 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.777357 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":74,"spinlock_wait_cycles":1408,"thread_start_us":61,"threads_started":1}
I20260925 14:44:58.777469 24644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.773609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.21.205.195:42355)
0925 14:44:58.773747 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.773759 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.773832 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.773991 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.773992 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.773996 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:44:58.774109 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.774111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.775038 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.775038 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.776411 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.776414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.777135 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.777135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.777233 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.777242 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.777301 (+ 59us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.777357 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":134,"thread_start_us":56,"threads_started":1}
I20260925 14:44:58.777969 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6363cb4bf1ec4e359b750ab09cbc3e80" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0" is_pre_election: true
I20260925 14:44:58.778008 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6363cb4bf1ec4e359b750ab09cbc3e80" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea" is_pre_election: true
I20260925 14:44:58.778081 24228 raft_consensus.cc:2454] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:44:58.778090 24379 raft_consensus.cc:2454] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:44:58.778249 24451 leader_election.cc:302] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:44:58.778388 24597 raft_consensus.cc:2781] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:44:58.778426 24597 raft_consensus.cc:482] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:44:58.778453 24597 raft_consensus.cc:3037] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.779209 24597 raft_consensus.cc:504] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.779343 24597 leader_election.cc:288] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Requested vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:44:58.779486 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6363cb4bf1ec4e359b750ab09cbc3e80" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea"
I20260925 14:44:58.779497 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6363cb4bf1ec4e359b750ab09cbc3e80" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0"
I20260925 14:44:58.779563 24228 raft_consensus.cc:3037] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.779583 24379 raft_consensus.cc:3037] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:44:58.780247 24228 raft_consensus.cc:2454] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:44:58.780261 24379 raft_consensus.cc:2454] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:44:58.780395 24451 leader_election.cc:302] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:44:58.780490 24597 raft_consensus.cc:2781] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:44:58.780591 24597 raft_consensus.cc:686] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Becoming Leader. State: Replica: 75b350545a904dc2bae94677003887d0, State: Running, Role: LEADER
I20260925 14:44:58.780652 24597 consensus_queue.cc:237] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:44:58.781322 24079 catalog_manager.cc:5495] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b350545a904dc2bae94677003887d0 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "75b350545a904dc2bae94677003887d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } health_report { overall_health: HEALTHY } } }
I20260925 14:44:58.804773 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.802729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.21.205.195:40993)
0925 14:44:58.802768 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.802770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.802778 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:44:58.802847 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.802848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.802864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.802915 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.803004 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.803450 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.804181 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.804354 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.804406 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.804526 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.804622 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.804674 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.804708 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.804724 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:44:58.809048 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.806969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:60641 (local address 127.21.205.193:46425)
0925 14:44:58.807042 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.807043 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.807120 (+ 77us) server_negotiation.cc:407] Connection header received
0925 14:44:58.807155 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.807156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.807163 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.807210 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.807342 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.807812 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.808502 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.808686 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.808734 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.808884 (+ 150us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.808907 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.808942 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.808989 (+ 47us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.809004 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:44:58.809075 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.806891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.21.205.195:60641)
0925 14:44:58.807054 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.807064 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.807118 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.807229 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.807230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.807231 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:58.807293 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.807294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.807881 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.807882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.808435 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.808437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.808750 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.808751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.808796 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.808808 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.808955 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.808956 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.808989 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.809017 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260925 14:44:58.811198 24379 raft_consensus.cc:1261] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.811228 24228 raft_consensus.cc:1261] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:44:58.811460 24597 consensus_queue.cc:1035] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.811846 24646 consensus_queue.cc:1035] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:44:58.813097 24619 mvcc.cc:205] Tried to move back new op lower bound from 7333263355129217024 to 7333263355006091264. Current Snapshot: MvccSnapshot[applied={T|T < 7333263355129217024}]
I20260925 14:44:58.818495 24586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.816391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:48367 (local address 127.21.205.195:40993)
0925 14:44:58.816427 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 14:44:58.816429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:44:58.816438 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:44:58.816488 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:44:58.816488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:44:58.816498 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:44:58.816551 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:44:58.816703 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.817199 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.817849 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.818015 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.818093 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.818268 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.818301 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.818350 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:44:58.818412 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 14:44:58.818429 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:44:58.818533 24634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:44:58.816353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40993 (local address 127.21.205.193:48367)
0925 14:44:58.816395 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 14:44:58.816404 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:44:58.816438 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:44:58.816589 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:44:58.816590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:44:58.816591 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:44:58.816627 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.816628 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.817268 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.817268 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.817776 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:44:58.817777 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:44:58.818110 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:44:58.818111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:44:58.818183 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:44:58.818190 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:44:58.818363 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:44:58.818365 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:44:58.818408 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 14:44:58.818459 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
W20260925 14:44:58.818811 24619 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 14:45:00.337137 24078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:00.337260 24078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50564:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:00.338011 24078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Sending DeleteTablet for 3 replicas of tablet 6363cb4bf1ec4e359b750ab09cbc3e80
I20260925 14:45:00.338268 24500 tablet_service.cc:1514] Processing DeleteTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260925 14:45:00.338256 24359 tablet_service.cc:1514] Processing DeleteTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:57772
I20260925 14:45:00.338254 24218 tablet_service.cc:1514] Processing DeleteTablet for tablet 6363cb4bf1ec4e359b750ab09cbc3e80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32920
I20260925 14:45:00.338541 24679 tablet_replica.cc:330] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: stopping tablet replica
I20260925 14:45:00.338593 24679 raft_consensus.cc:2229] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:00.338649 24679 raft_consensus.cc:2258] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.338738 24677 tablet_replica.cc:330] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: stopping tablet replica
I20260925 14:45:00.338786 24677 raft_consensus.cc:2229] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:00.338865 24677 raft_consensus.cc:2258] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.339049 24679 ts_tablet_manager.cc:1905] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.340082 24677 ts_tablet_manager.cc:1905] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.342150 24677 ts_tablet_manager.cc:1918] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 14:45:00.342216 24677 log.cc:1196] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/6363cb4bf1ec4e359b750ab09cbc3e80
I20260925 14:45:00.342437 24679 ts_tablet_manager.cc:1918] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 14:45:00.342489 24677 ts_tablet_manager.cc:1939] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 75b350545a904dc2bae94677003887d0: Deleting consensus metadata
I20260925 14:45:00.342500 24679 log.cc:1196] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/6363cb4bf1ec4e359b750ab09cbc3e80
I20260925 14:45:00.342703 24679 ts_tablet_manager.cc:1939] T 6363cb4bf1ec4e359b750ab09cbc3e80 P 8d75935ed62e4e158ba4102541b16ec0: Deleting consensus metadata
I20260925 14:45:00.342803 24066 catalog_manager.cc:4841] TS 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993): tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (table test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]) successfully deleted
I20260925 14:45:00.342965 24066 catalog_manager.cc:4841] TS 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947): tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (table test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]) successfully deleted
I20260925 14:45:00.343011 24678 tablet_replica.cc:330] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: stopping tablet replica
I20260925 14:45:00.343081 24678 raft_consensus.cc:2229] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:00.343127 24678 raft_consensus.cc:2258] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.343343 24678 ts_tablet_manager.cc:1905] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.344870 24678 ts_tablet_manager.cc:1918] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 14:45:00.344920 24678 log.cc:1196] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/6363cb4bf1ec4e359b750ab09cbc3e80
I20260925 14:45:00.345081 24678 ts_tablet_manager.cc:1939] T 6363cb4bf1ec4e359b750ab09cbc3e80 P a0825ee4b4394c0d84229620848b07ea: Deleting consensus metadata
I20260925 14:45:00.345641 24063 catalog_manager.cc:4841] TS a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425): tablet 6363cb4bf1ec4e359b750ab09cbc3e80 (table test-table [id=d0a28c769fb54f0b9be067952bc9bbf3]) successfully deleted
Sep 25 14:45:00 dist-test-slave-d7lw krb5kdc[24033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 14:45:00 dist-test-slave-d7lw krb5kdc[24033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347500, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:00.360975 24687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:00.354190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50588 (local address 127.21.205.254:38521)
0925 14:45:00.354323 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:00.354325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:00.354364 (+ 39us) server_negotiation.cc:407] Connection header received
0925 14:45:00.354435 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:00.354436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:00.354445 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:00.354506 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:00.355064 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:00.355579 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:00.356352 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:00.356584 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:00.356634 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:00.358662 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:00.358664 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:00.358664 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:00.358670 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:00.359997 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:00.360327 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:00.360327 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:00.360328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:00.360375 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:00.360547 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:00.360547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:00.360548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:00.360641 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:00.360715 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:00.360853 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 14:45:00.360911 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":82,"threads_started":1}
I20260925 14:45:00.362479 24078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:00.362608 24078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:00.364043 24359 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d (DEFAULT_TABLE table=test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.364068 24500 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d (DEFAULT_TABLE table=test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.364078 24218 tablet_service.cc:1467] Processing CreateTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d (DEFAULT_TABLE table=test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.364173 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9aa73ad3e14a50b440ef77fd870d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.364172 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9aa73ad3e14a50b440ef77fd870d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.364172 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b9aa73ad3e14a50b440ef77fd870d3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.365108 24688 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Bootstrap starting.
I20260925 14:45:00.365515 24688 tablet_bootstrap.cc:654] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.365579 24689 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Bootstrap starting.
I20260925 14:45:00.365922 24689 tablet_bootstrap.cc:654] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.366083 24688 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: No bootstrap required, opened a new log
I20260925 14:45:00.366137 24688 ts_tablet_manager.cc:1397] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:00.366233 24690 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Bootstrap starting.
I20260925 14:45:00.366255 24688 raft_consensus.cc:348] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.366345 24688 raft_consensus.cc:374] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.366371 24688 raft_consensus.cc:729] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.366426 24688 consensus_queue.cc:260] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.366535 24688 ts_tablet_manager.cc:1428] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:00.366724 24690 tablet_bootstrap.cc:654] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.367305 24689 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: No bootstrap required, opened a new log
I20260925 14:45:00.367406 24689 ts_tablet_manager.cc:1397] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:00.367522 24690 tablet_bootstrap.cc:492] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: No bootstrap required, opened a new log
I20260925 14:45:00.367573 24690 ts_tablet_manager.cc:1397] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:00.367529 24689 raft_consensus.cc:348] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.367599 24689 raft_consensus.cc:374] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.367632 24689 raft_consensus.cc:729] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.367699 24689 consensus_queue.cc:260] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.367684 24690 raft_consensus.cc:348] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.367751 24690 raft_consensus.cc:374] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.367797 24690 raft_consensus.cc:729] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b350545a904dc2bae94677003887d0, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.367895 24689 ts_tablet_manager.cc:1428] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:00.367873 24690 consensus_queue.cc:260] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.368333 24690 ts_tablet_manager.cc:1428] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:00.429167 24671 raft_consensus.cc:482] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:00.429299 24671 raft_consensus.cc:504] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.429528 24671 leader_election.cc:288] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:45:00.429714 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9aa73ad3e14a50b440ef77fd870d3d" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea" is_pre_election: true
I20260925 14:45:00.429714 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9aa73ad3e14a50b440ef77fd870d3d" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0" is_pre_election: true
I20260925 14:45:00.429827 24379 raft_consensus.cc:2454] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:45:00.429827 24228 raft_consensus.cc:2454] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:45:00.429994 24454 leader_election.cc:302] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, 8d75935ed62e4e158ba4102541b16ec0; no voters:
I20260925 14:45:00.430126 24671 raft_consensus.cc:2781] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:00.430181 24671 raft_consensus.cc:482] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:00.430210 24671 raft_consensus.cc:3037] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.430956 24671 raft_consensus.cc:504] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.431124 24671 leader_election.cc:288] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Requested vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:45:00.431262 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9aa73ad3e14a50b440ef77fd870d3d" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea"
I20260925 14:45:00.431268 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b9aa73ad3e14a50b440ef77fd870d3d" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0"
I20260925 14:45:00.431349 24379 raft_consensus.cc:3037] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.431350 24228 raft_consensus.cc:3037] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.432161 24379 raft_consensus.cc:2454] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:45:00.432161 24228 raft_consensus.cc:2454] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:45:00.432368 24454 leader_election.cc:302] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, 8d75935ed62e4e158ba4102541b16ec0; no voters:
I20260925 14:45:00.432507 24671 raft_consensus.cc:2781] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:00.432588 24671 raft_consensus.cc:686] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Becoming Leader. State: Replica: 75b350545a904dc2bae94677003887d0, State: Running, Role: LEADER
I20260925 14:45:00.432639 24671 consensus_queue.cc:237] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.433223 24078 catalog_manager.cc:5495] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b350545a904dc2bae94677003887d0 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "75b350545a904dc2bae94677003887d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } health_report { overall_health: HEALTHY } } }
I20260925 14:45:00.442175 24697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:00.440171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.21.205.195:40993)
0925 14:45:00.440330 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:00.440332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:00.440339 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:00.440357 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:00.440357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:00.440367 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:00.440420 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:00.440509 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:00.440959 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:00.441621 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:00.441804 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:00.441856 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:00.441970 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:00.441994 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:00.442026 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:00.442065 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 14:45:00.442084 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260925 14:45:00.444130 24228 raft_consensus.cc:1261] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:00.444159 24379 raft_consensus.cc:1261] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:00.444308 24646 consensus_queue.cc:1035] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:00.444378 24671 consensus_queue.cc:1035] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:00.453251 24078 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:00.453338 24078 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:00.453881 24078 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Sending DeleteTablet for 3 replicas of tablet 5b9aa73ad3e14a50b440ef77fd870d3d
I20260925 14:45:00.454080 24218 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32920
I20260925 14:45:00.454142 24500 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260925 14:45:00.454188 24678 tablet_replica.cc:330] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: stopping tablet replica
I20260925 14:45:00.454183 24359 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b9aa73ad3e14a50b440ef77fd870d3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:00 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:57772
I20260925 14:45:00.454252 24678 raft_consensus.cc:2229] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:00.454248 24677 tablet_replica.cc:330] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: stopping tablet replica
I20260925 14:45:00.454262 24679 tablet_replica.cc:330] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: stopping tablet replica
I20260925 14:45:00.454296 24678 raft_consensus.cc:2258] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.454303 24677 raft_consensus.cc:2229] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:00.454306 24679 raft_consensus.cc:2229] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:00.454340 24679 raft_consensus.cc:2258] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.454348 24677 raft_consensus.cc:2258] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:00.454656 24678 ts_tablet_manager.cc:1905] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.454699 24679 ts_tablet_manager.cc:1905] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.455674 24677 ts_tablet_manager.cc:1905] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:00.455813 24078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:00.455961 24078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:00.456032 24679 ts_tablet_manager.cc:1918] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:00.456089 24679 log.cc:1196] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/5b9aa73ad3e14a50b440ef77fd870d3d
I20260925 14:45:00.456307 24679 ts_tablet_manager.cc:1939] T 5b9aa73ad3e14a50b440ef77fd870d3d P 8d75935ed62e4e158ba4102541b16ec0: Deleting consensus metadata
I20260925 14:45:00.456741 24066 catalog_manager.cc:4841] TS 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947): tablet 5b9aa73ad3e14a50b440ef77fd870d3d (table test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]) successfully deleted
I20260925 14:45:00.457155 24359 tablet_service.cc:1467] Processing CreateTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 (DEFAULT_TABLE table=test-table [id=a87ce918d6754814834323d1f34f0555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.457195 24500 tablet_service.cc:1467] Processing CreateTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 (DEFAULT_TABLE table=test-table [id=a87ce918d6754814834323d1f34f0555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.457261 24359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860dfc238f5409fb7e4704b6fe515f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.457376 24500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860dfc238f5409fb7e4704b6fe515f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.457454 24218 tablet_service.cc:1467] Processing CreateTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 (DEFAULT_TABLE table=test-table [id=a87ce918d6754814834323d1f34f0555]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:00.457598 24678 ts_tablet_manager.cc:1918] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:00.457605 24218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c860dfc238f5409fb7e4704b6fe515f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:00.457654 24678 log.cc:1196] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/5b9aa73ad3e14a50b440ef77fd870d3d
I20260925 14:45:00.457890 24678 ts_tablet_manager.cc:1939] T 5b9aa73ad3e14a50b440ef77fd870d3d P a0825ee4b4394c0d84229620848b07ea: Deleting consensus metadata
I20260925 14:45:00.458194 24063 catalog_manager.cc:4841] TS a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425): tablet 5b9aa73ad3e14a50b440ef77fd870d3d (table test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]) successfully deleted
I20260925 14:45:00.458475 24689 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: Bootstrap starting.
I20260925 14:45:00.458505 24690 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Bootstrap starting.
I20260925 14:45:00.458760 24690 tablet_bootstrap.cc:654] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.458772 24689 tablet_bootstrap.cc:654] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.458957 24688 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: Bootstrap starting.
I20260925 14:45:00.457378 24677 ts_tablet_manager.cc:1918] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:00.459138 24677 log.cc:1196] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/5b9aa73ad3e14a50b440ef77fd870d3d
I20260925 14:45:00.459275 24688 tablet_bootstrap.cc:654] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:00.459316 24677 ts_tablet_manager.cc:1939] T 5b9aa73ad3e14a50b440ef77fd870d3d P 75b350545a904dc2bae94677003887d0: Deleting consensus metadata
I20260925 14:45:00.459578 24066 catalog_manager.cc:4841] TS 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993): tablet 5b9aa73ad3e14a50b440ef77fd870d3d (table test-table [id=7dbe2da5b82d4cdd82a3da2cc9f1d710]) successfully deleted
I20260925 14:45:00.459595 24690 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: No bootstrap required, opened a new log
I20260925 14:45:00.459640 24690 ts_tablet_manager.cc:1397] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:00.459743 24690 raft_consensus.cc:348] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.459836 24690 raft_consensus.cc:374] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.459865 24690 raft_consensus.cc:729] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75b350545a904dc2bae94677003887d0, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.459870 24688 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: No bootstrap required, opened a new log
I20260925 14:45:00.459906 24688 ts_tablet_manager.cc:1397] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:00.459921 24690 consensus_queue.cc:260] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.460004 24690 ts_tablet_manager.cc:1428] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:00.459991 24688 raft_consensus.cc:348] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.460049 24688 raft_consensus.cc:374] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.460067 24688 raft_consensus.cc:729] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0825ee4b4394c0d84229620848b07ea, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.460119 24688 consensus_queue.cc:260] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.460199 24688 ts_tablet_manager.cc:1428] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:00.460768 24689 tablet_bootstrap.cc:492] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: No bootstrap required, opened a new log
I20260925 14:45:00.460819 24689 ts_tablet_manager.cc:1397] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 14:45:00.460907 24689 raft_consensus.cc:348] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.460945 24689 raft_consensus.cc:374] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:00.460958 24689 raft_consensus.cc:729] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d75935ed62e4e158ba4102541b16ec0, State: Initialized, Role: FOLLOWER
I20260925 14:45:00.460988 24689 consensus_queue.cc:260] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.461041 24689 ts_tablet_manager.cc:1428] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:00.622493 24668 raft_consensus.cc:482] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:00.622584 24668 raft_consensus.cc:504] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.622768 24668 leader_election.cc:288] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:45:00.622997 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860dfc238f5409fb7e4704b6fe515f6" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0" is_pre_election: true
I20260925 14:45:00.622998 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860dfc238f5409fb7e4704b6fe515f6" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea" is_pre_election: true
I20260925 14:45:00.623093 24228 raft_consensus.cc:2454] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:45:00.623093 24379 raft_consensus.cc:2454] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 0.
I20260925 14:45:00.623238 24451 leader_election.cc:302] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, a0825ee4b4394c0d84229620848b07ea; no voters:
I20260925 14:45:00.623358 24668 raft_consensus.cc:2781] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:00.623413 24668 raft_consensus.cc:482] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:00.623428 24668 raft_consensus.cc:3037] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.624136 24668 raft_consensus.cc:504] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.624282 24668 leader_election.cc:288] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Requested vote from peers a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425), 8d75935ed62e4e158ba4102541b16ec0 (127.21.205.194:42947)
I20260925 14:45:00.624431 24379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860dfc238f5409fb7e4704b6fe515f6" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d75935ed62e4e158ba4102541b16ec0"
I20260925 14:45:00.624431 24228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c860dfc238f5409fb7e4704b6fe515f6" candidate_uuid: "75b350545a904dc2bae94677003887d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0825ee4b4394c0d84229620848b07ea"
I20260925 14:45:00.624497 24379 raft_consensus.cc:3037] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.624516 24228 raft_consensus.cc:3037] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:00.625077 24379 raft_consensus.cc:2454] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:45:00.625185 24228 raft_consensus.cc:2454] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75b350545a904dc2bae94677003887d0 in term 1.
I20260925 14:45:00.625277 24454 leader_election.cc:302] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75b350545a904dc2bae94677003887d0, 8d75935ed62e4e158ba4102541b16ec0; no voters:
I20260925 14:45:00.625365 24668 raft_consensus.cc:2781] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:00.625427 24668 raft_consensus.cc:686] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Becoming Leader. State: Replica: 75b350545a904dc2bae94677003887d0, State: Running, Role: LEADER
I20260925 14:45:00.625511 24668 consensus_queue.cc:237] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } }
I20260925 14:45:00.626165 24077 catalog_manager.cc:5495] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75b350545a904dc2bae94677003887d0 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "75b350545a904dc2bae94677003887d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75b350545a904dc2bae94677003887d0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40993 } health_report { overall_health: HEALTHY } } }
I20260925 14:45:00.644079 24228 raft_consensus.cc:1261] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:00.644079 24379 raft_consensus.cc:1261] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Refusing update from remote peer 75b350545a904dc2bae94677003887d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:00.644310 24646 consensus_queue.cc:1035] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0825ee4b4394c0d84229620848b07ea" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:00.644394 24668 consensus_queue.cc:1035] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d75935ed62e4e158ba4102541b16ec0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 14:45:00.649277 24659 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 14:45:02.165573 24077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:02.165685 24077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:02.166538 24077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79f7e1ab9e8944c3a92f77b5d815212f: Sending DeleteTablet for 3 replicas of tablet c860dfc238f5409fb7e4704b6fe515f6
I20260925 14:45:02.166782 24218 tablet_service.cc:1514] Processing DeleteTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:02 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32920
I20260925 14:45:02.166819 24359 tablet_service.cc:1514] Processing DeleteTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:02 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:57772
I20260925 14:45:02.166847 24500 tablet_service.cc:1514] Processing DeleteTablet for tablet c860dfc238f5409fb7e4704b6fe515f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:02 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49190
I20260925 14:45:02.166999 24728 tablet_replica.cc:330] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea: stopping tablet replica
I20260925 14:45:02.167083 24728 raft_consensus.cc:2229] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:02.167125 24728 raft_consensus.cc:2258] T c860dfc238f5409fb7e4704b6fe515f6 P a0825ee4b4394c0d84229620848b07ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:02.167109 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24151
I20260925 14:45:02.167102 24730 tablet_replica.cc:330] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: stopping tablet replica
I20260925 14:45:02.167973 24730 raft_consensus.cc:2229] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:02.168035 24730 raft_consensus.cc:2258] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:02.168282 24730 ts_tablet_manager.cc:1905] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:02.170230 24730 ts_tablet_manager.cc:1918] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 14:45:02.170281 24730 log.cc:1196] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/c860dfc238f5409fb7e4704b6fe515f6
I20260925 14:45:02.170537 24730 ts_tablet_manager.cc:1939] T c860dfc238f5409fb7e4704b6fe515f6 P 75b350545a904dc2bae94677003887d0: Deleting consensus metadata
I20260925 14:45:02.170940 24066 catalog_manager.cc:4841] TS 75b350545a904dc2bae94677003887d0 (127.21.205.195:40993): tablet c860dfc238f5409fb7e4704b6fe515f6 (table test-table [id=a87ce918d6754814834323d1f34f0555]) successfully deleted
I20260925 14:45:02.171876 24729 tablet_replica.cc:330] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: stopping tablet replica
I20260925 14:45:02.171945 24729 raft_consensus.cc:2229] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:02.171983 24729 raft_consensus.cc:2258] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 14:45:02.175351 24063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:46425 (error 108)
I20260925 14:45:02.175462 24729 ts_tablet_manager.cc:1905] T c860dfc238f5409fb7e4704b6fe515f6 P 8d75935ed62e4e158ba4102541b16ec0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:02.175594 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24293
I20260925 14:45:02.176285 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:02.175723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46425 (local address 127.0.0.1:32936)
0925 14:45:02.176190 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:02.176238 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:46425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":73,"threads_started":1}
W20260925 14:45:02.176367 24063 catalog_manager.cc:4584] TS a0825ee4b4394c0d84229620848b07ea (127.21.205.193:46425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c860dfc238f5409fb7e4704b6fe515f6: Network error: Client connection negotiation failed: client connection to 127.21.205.193:46425: connect: Connection refused (error 111)
I20260925 14:45:02.181162 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24434
I20260925 14:45:02.181205 24733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:02.181073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42947 (local address 127.0.0.1:57792)
0925 14:45:02.181153 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:02.181177 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.194:42947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 14:45:02.186463 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24046
2026-09-25T14:45:02Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5929 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:02 dist-test-slave-d7lw krb5kdc[24736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:02 dist-test-slave-d7lw krb5kdc[24736](info): set up 2 sockets
Sep 25 14:45:02 dist-test-slave-d7lw krb5kdc[24736](info): commencing operation
krb5kdc: starting...
W20260925 14:45:03.227476 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:03.342264 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35879
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35879
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:03.420375 24749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:03.420557 24749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:03.420578 24749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:03.421702 24749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:03.421743 24749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:03.421757 24749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:03.421770 24749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:03.421819 24749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:03.423234 24749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:03.423458 24749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:03.423697 24749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:03.428511 24754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.428548 24749 server_base.cc:1034] running on GCE node
W20260925 14:45:03.428697 24756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.428619 24758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.429037 24749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:03.429327 24749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:03.430501 24749 hybrid_clock.cc:648] HybridClock initialized: now 1790347503430496 us; error 52 us; skew 500 ppm
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:03.432860 24749 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:03.433266 24749 webserver.cc:460] Webserver started at http://127.21.205.254:37239/ using document root <none> and password file <none>
I20260925 14:45:03.433496 24749 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:03.433562 24749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:03.433712 24749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:03.434710 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "09442790a7414f6f988dc626caba6d0d"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "706ad28db78489969b3dc76b23d0e579"
server_key_iv: "0b00a92410c7517c0a9f7104e2699420"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.435067 24749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "09442790a7414f6f988dc626caba6d0d"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "706ad28db78489969b3dc76b23d0e579"
server_key_iv: "0b00a92410c7517c0a9f7104e2699420"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.436487 24749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:03.437150 24765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.437317 24749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:03.437420 24749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "09442790a7414f6f988dc626caba6d0d"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "706ad28db78489969b3dc76b23d0e579"
server_key_iv: "0b00a92410c7517c0a9f7104e2699420"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.437491 24749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:03.444039 24749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:03.445693 24749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:03.445860 24749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:03.449695 24749 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:35879
I20260925 14:45:03.449728 24818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:35879 every 8 connection(s)
I20260925 14:45:03.450066 24749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:03.450909 24819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:03.453146 24819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d: Bootstrap starting.
I20260925 14:45:03.453711 24819 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:03.453984 24819 log.cc:824] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:03.454617 24819 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d: No bootstrap required, opened a new log
I20260925 14:45:03.456055 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24749
I20260925 14:45:03.456157 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:03.456333 22327 external_mini_cluster.cc:1428] Setting key 5a40f8a79daea3bcb117ed4109facf53
I20260925 14:45:03.456300 24819 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } }
I20260925 14:45:03.456453 24819 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:03.456527 24819 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09442790a7414f6f988dc626caba6d0d, State: Initialized, Role: FOLLOWER
I20260925 14:45:03.456720 24819 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } }
I20260925 14:45:03.456823 24819 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:03.456876 24819 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:03.456943 24819 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:03.457446 24819 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } }
I20260925 14:45:03.457561 24819 leader_election.cc:302] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09442790a7414f6f988dc626caba6d0d; no voters:
I20260925 14:45:03.457732 24819 leader_election.cc:288] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:03.457768 24824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:03.457887 24824 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [term 1 LEADER]: Becoming Leader. State: Replica: 09442790a7414f6f988dc626caba6d0d, State: Running, Role: LEADER
I20260925 14:45:03.458020 24819 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:03.458042 24824 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } }
I20260925 14:45:03.458503 24826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09442790a7414f6f988dc626caba6d0d. Latest consensus state: current_term: 1 leader_uuid: "09442790a7414f6f988dc626caba6d0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } } }
I20260925 14:45:03.458592 24826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:03.458803 24825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09442790a7414f6f988dc626caba6d0d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09442790a7414f6f988dc626caba6d0d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35879 } } }
I20260925 14:45:03.459112 24825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:03.459319 24833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:03.460500 24833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:03.467954 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.456900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56984 (local address 127.21.205.254:35879)
0925 14:45:03.457094 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:03.457103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:03.457133 (+ 30us) server_negotiation.cc:407] Connection header received
0925 14:45:03.457497 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:03.457504 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:03.457572 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:03.457705 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:03.458549 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.459817 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.460930 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.461516 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.461639 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.464230 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:03.464234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:03.464235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:03.464245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:03.466049 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.466383 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.466386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.466386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.466878 (+ 492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.467060 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.467061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.467061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.467357 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:03.467529 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:03.467594 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 14:45:03.467726 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260925 14:45:03.468210 24833 catalog_manager.cc:1347] Generated new cluster ID: edd3bc5f014d4dcea99d2c2cd6e67554
I20260925 14:45:03.468259 24833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:03.472702 24833 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:03.473160 24833 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:03.476466 24833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09442790a7414f6f988dc626caba6d0d: Generated new TSK 0
I20260925 14:45:03.476631 24833 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:03.504493 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:03.584208 24845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:03.584386 24845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:03.584408 24845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:03.585513 24845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:03.585548 24845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:03.585573 24845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:03.586895 24845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:03.587055 24845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:03.587208 24845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:03.592110 24854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.592141 24851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.592162 24852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.592404 24845 server_base.cc:1034] running on GCE node
I20260925 14:45:03.592622 24845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:03.592866 24845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:03.594002 24845 hybrid_clock.cc:648] HybridClock initialized: now 1790347503593993 us; error 27 us; skew 500 ppm
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:03.596333 24845 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:03.596681 24845 webserver.cc:460] Webserver started at http://127.21.205.193:32895/ using document root <none> and password file <none>
I20260925 14:45:03.596881 24845 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:03.596918 24845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:03.596997 24845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:03.597939 24845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "b5d503180a0f483585e7ff46155bc5e5"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "8f4bb515ec79e8021342be4e4cf72517"
server_key_iv: "8161fb8c23fa77d1fb85896ec4490f16"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.598227 24845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "b5d503180a0f483585e7ff46155bc5e5"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "8f4bb515ec79e8021342be4e4cf72517"
server_key_iv: "8161fb8c23fa77d1fb85896ec4490f16"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.599426 24845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:03.600128 24861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.600296 24845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.600383 24845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "b5d503180a0f483585e7ff46155bc5e5"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "8f4bb515ec79e8021342be4e4cf72517"
server_key_iv: "8161fb8c23fa77d1fb85896ec4490f16"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.600466 24845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:03.610598 24845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:03.612322 24845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:03.612468 24845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:03.612679 24845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:03.612957 24845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:03.613009 24845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.613049 24845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:03.613076 24845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.618520 24845 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:35569
I20260925 14:45:03.618611 24975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:35569 every 8 connection(s)
I20260925 14:45:03.618896 24845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:03.627097 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.619609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:57521 (local address 127.21.205.254:35879)
0925 14:45:03.619667 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:03.619669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:03.619891 (+ 222us) server_negotiation.cc:407] Connection header received
0925 14:45:03.620333 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:03.620333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:03.620347 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:03.620397 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:03.621217 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.621723 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.622663 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.622863 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.622907 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.624861 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:03.624863 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:03.624863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:03.624870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:03.626046 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.626377 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.626378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.626378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.626424 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.626615 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.626615 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.626616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.626713 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:03.626760 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:03.626977 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 14:45:03.627038 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 14:45:03.627367 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.619588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35879 (local address 127.21.205.193:57521)
0925 14:45:03.619743 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:03.619809 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:03.620247 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:03.620443 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:03.620446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:03.620957 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:03.621139 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.621144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.621799 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.621800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.622561 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.622562 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.622931 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.622931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.623025 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.623446 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:03.623454 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:03.624749 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:03.626114 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.626119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.626121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.626315 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.626477 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.626478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.626478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.626564 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.626764 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:03.626768 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:03.626865 (+ 97us) client_negotiation.cc:769] Sending connection context
0925 14:45:03.626949 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:45:03.627054 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260925 14:45:03.627942 24976 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35879
I20260925 14:45:03.628021 24976 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:03.628149 24976 heartbeater.cc:511] Master 127.21.205.254:35879 requested a full tablet report, sending...
I20260925 14:45:03.628590 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24845
I20260925 14:45:03.628605 24782 ts_manager.cc:194] Registered new tserver with Master: b5d503180a0f483585e7ff46155bc5e5 (127.21.205.193:35569)
I20260925 14:45:03.628657 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:03.628831 22327 external_mini_cluster.cc:1428] Setting key a5619f3fc653c2283968946466dd0f3d
I20260925 14:45:03.629417 24782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:57521
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:03.652244 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:03.729588 24982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:03.729760 24982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:03.729782 24982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:03.730896 24982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:03.730934 24982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:03.730983 24982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:03.732424 24982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:03.732676 24982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:03.732874 24982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:03.737713 24990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.737789 24982 server_base.cc:1034] running on GCE node
W20260925 14:45:03.737759 24988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.737742 24987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.738123 24982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:03.738379 24982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:03.739540 24982 hybrid_clock.cc:648] HybridClock initialized: now 1790347503739526 us; error 26 us; skew 500 ppm
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:03.742133 24982 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:03.742535 24982 webserver.cc:460] Webserver started at http://127.21.205.194:37573/ using document root <none> and password file <none>
I20260925 14:45:03.742753 24982 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:03.742820 24982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:03.742966 24982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:03.744304 24982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "e9c702aa509e445fb2928cf92c1dcfdb"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "b46f2a587f3c10b8333df3548b34a36b"
server_key_iv: "b238f671d6c8b58a32a70976457e343f"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.744776 24982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "e9c702aa509e445fb2928cf92c1dcfdb"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "b46f2a587f3c10b8333df3548b34a36b"
server_key_iv: "b238f671d6c8b58a32a70976457e343f"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.746668 24982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260925 14:45:03.747675 24997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.747866 24982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:03.747956 24982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "e9c702aa509e445fb2928cf92c1dcfdb"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "b46f2a587f3c10b8333df3548b34a36b"
server_key_iv: "b238f671d6c8b58a32a70976457e343f"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.748023 24982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:03.755625 24982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:03.757820 24982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:03.757972 24982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:03.758206 24982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:03.758441 24982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:03.758471 24982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.758503 24982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:03.758515 24982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.764261 24982 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:44173
I20260925 14:45:03.764315 25111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:44173 every 8 connection(s)
I20260925 14:45:03.764586 24982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:03.766191 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 24982
I20260925 14:45:03.766301 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:03.766533 22327 external_mini_cluster.cc:1428] Setting key 9e45007255163a921917d97ea11e8941
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:03.774466 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.765284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:57545 (local address 127.21.205.254:35879)
0925 14:45:03.765347 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:03.765349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:03.765512 (+ 163us) server_negotiation.cc:407] Connection header received
0925 14:45:03.765953 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:03.765954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:03.765962 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:03.766015 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:03.767236 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.768048 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.768999 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.769334 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.769412 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.771743 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:03.771745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:03.771745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:03.771752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:03.773065 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.773502 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.773504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.773504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.773566 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.773817 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.773818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.773819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.773938 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:03.774018 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:03.774290 (+ 272us) server_negotiation.cc:299] Negotiation successful
0925 14:45:03.774375 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:45:03.774669 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.765242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35879 (local address 127.21.205.194:57545)
0925 14:45:03.765390 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:03.765428 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:03.765871 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:03.766124 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:03.766129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:03.766884 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:03.767141 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.767147 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.768132 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.768133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.768912 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.768914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.769421 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.769422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.769518 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.770004 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:03.770012 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:03.771643 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:03.773173 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.773180 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.773182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.773410 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.773640 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.773641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.773641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.773748 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.774021 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:03.774028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:03.774161 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 14:45:03.774268 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 14:45:03.774427 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260925 14:45:03.775103 25112 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35879
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
I20260925 14:45:03.775192 25112 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:03.775346 25112 heartbeater.cc:511] Master 127.21.205.254:35879 requested a full tablet report, sending...
I20260925 14:45:03.775681 24782 ts_manager.cc:194] Registered new tserver with Master: e9c702aa509e445fb2928cf92c1dcfdb (127.21.205.194:44173)
I20260925 14:45:03.776105 24782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:57545
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:03.790292 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:38821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:03.869791 25118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:03.870002 25118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:03.870035 25118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:03.871147 25118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:03.871210 25118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:03.871254 25118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:03.872658 25118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:38821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:03.872897 25118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:03.873122 25118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:03.878005 25124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.877979 25123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:03.878028 25126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:03.878090 25118 server_base.cc:1034] running on GCE node
I20260925 14:45:03.878381 25118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:03.878621 25118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:03.879751 25118 hybrid_clock.cc:648] HybridClock initialized: now 1790347503879746 us; error 27 us; skew 500 ppm
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:03.882035 25118 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:03.882463 25118 webserver.cc:460] Webserver started at http://127.21.205.195:36821/ using document root <none> and password file <none>
I20260925 14:45:03.882697 25118 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:03.882764 25118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:03.882911 25118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:03.883945 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "4ea91bb21957465995704899c6a8918c"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "111d6896f842148280c3cc637512ee69"
server_key_iv: "be47ba4b907a1f461603d69fc2c3899c"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.884284 25118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "4ea91bb21957465995704899c6a8918c"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "111d6896f842148280c3cc637512ee69"
server_key_iv: "be47ba4b907a1f461603d69fc2c3899c"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.885462 25118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:03.886171 25133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.886343 25118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.886435 25118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "4ea91bb21957465995704899c6a8918c"
format_stamp: "Formatted at 2026-09-25 14:45:03 on dist-test-slave-d7lw"
server_key: "111d6896f842148280c3cc637512ee69"
server_key_iv: "be47ba4b907a1f461603d69fc2c3899c"
server_key_version: "encryptionkey@0"
I20260925 14:45:03.886514 25118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:03.901180 25118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:03.902855 25118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:03.903009 25118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:03.903226 25118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:03.903522 25118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:03.903576 25118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.903620 25118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:03.903648 25118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:03.909225 25118 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36093
I20260925 14:45:03.909272 25247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36093 every 8 connection(s)
I20260925 14:45:03.909529 25118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:45:03.914170 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25118
I20260925 14:45:03.914268 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:03.914476 22327 external_mini_cluster.cc:1428] Setting key 3b3742bcd2683ea8aae9e6495f38c443
Sep 25 14:45:03 dist-test-slave-d7lw krb5kdc[24736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347503, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:03.918260 24823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.910142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:40643 (local address 127.21.205.254:35879)
0925 14:45:03.910202 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:03.910204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:03.910387 (+ 183us) server_negotiation.cc:407] Connection header received
0925 14:45:03.910867 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:03.910869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:03.910882 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:03.910945 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:03.911840 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.912319 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.913344 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.913562 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.913639 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.915951 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:03.915952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:03.915952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:03.915958 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:03.917155 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.917487 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.917488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.917488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.917537 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:03.917756 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:03.917756 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:03.917756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:03.917868 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:03.917916 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:03.918143 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 14:45:03.918199 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:45:03.918448 25250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:03.910133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35879 (local address 127.21.205.195:40643)
0925 14:45:03.910273 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:03.910303 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:03.910744 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:03.911017 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:03.911020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:03.911544 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:03.911720 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.911724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.912407 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.912408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.913219 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:03.913222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:03.913658 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:03.913659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:03.913775 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:03.914249 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:03.914258 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:03.915869 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:03.917224 (+ 1355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.917229 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.917231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.917424 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.917591 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:03.917592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:03.917592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:03.917704 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:03.917930 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:03.917934 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:03.918035 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 14:45:03.918109 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:45:03.918222 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260925 14:45:03.918910 25248 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35879
I20260925 14:45:03.918998 25248 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:03.919123 25248 heartbeater.cc:511] Master 127.21.205.254:35879 requested a full tablet report, sending...
I20260925 14:45:03.919438 24782 ts_manager.cc:194] Registered new tserver with Master: 4ea91bb21957465995704899c6a8918c (127.21.205.195:36093)
I20260925 14:45:03.919754 24782 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:40643
I20260925 14:45:03.926218 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 14:45:03.937122 24823 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:03.936304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56992 (local address 127.21.205.254:35879)
0925 14:45:03.936375 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:03.936377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:03.936456 (+ 79us) server_negotiation.cc:407] Connection header received
0925 14:45:03.936498 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:03.936498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:03.936506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:03.936567 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:03.937046 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56992: BlockingRecv error: recv got EOF from 127.0.0.1:56992 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:45:03.937963 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24845
I20260925 14:45:03.942833 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24982
I20260925 14:45:03.947016 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25118
I20260925 14:45:03.951143 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 24749
2026-09-25T14:45:03Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1763 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:04 dist-test-slave-d7lw krb5kdc[25260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:04 dist-test-slave-d7lw krb5kdc[25260](info): set up 2 sockets
Sep 25 14:45:04 dist-test-slave-d7lw krb5kdc[25260](info): commencing operation
krb5kdc: starting...
W20260925 14:45:04.966612 22327 mini_kdc.cc:121] Time spent starting KDC: real 0.996s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:05.075539 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42029
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:05.152983 25273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:05.153196 25273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:05.153234 25273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:05.154349 25273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:05.154408 25273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:05.154433 25273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:05.154461 25273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:05.154488 25273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:05.155944 25273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:05.156198 25273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:05.156399 25273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:05.161015 25279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:05.161039 25278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:05.161046 25281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:05.161458 25273 server_base.cc:1034] running on GCE node
I20260925 14:45:05.161671 25273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:05.161885 25273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:05.163028 25273 hybrid_clock.cc:648] HybridClock initialized: now 1790347505163012 us; error 29 us; skew 500 ppm
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347505, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:05.165380 25273 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:05.165751 25273 webserver.cc:460] Webserver started at http://127.21.205.254:42591/ using document root <none> and password file <none>
I20260925 14:45:05.165930 25273 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:05.165967 25273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:05.166045 25273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:05.166936 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d"
format_stamp: "Formatted at 2026-09-25 14:45:05 on dist-test-slave-d7lw"
server_key: "b4a3577d78f0448f0dbdf69939b6ecd3"
server_key_iv: "363d04f7f726b71a1bee8a065fa0159f"
server_key_version: "encryptionkey@0"
I20260925 14:45:05.167212 25273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d"
format_stamp: "Formatted at 2026-09-25 14:45:05 on dist-test-slave-d7lw"
server_key: "b4a3577d78f0448f0dbdf69939b6ecd3"
server_key_iv: "363d04f7f726b71a1bee8a065fa0159f"
server_key_version: "encryptionkey@0"
I20260925 14:45:05.168478 25273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:05.169199 25288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:05.169384 25273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:05.169474 25273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d"
format_stamp: "Formatted at 2026-09-25 14:45:05 on dist-test-slave-d7lw"
server_key: "b4a3577d78f0448f0dbdf69939b6ecd3"
server_key_iv: "363d04f7f726b71a1bee8a065fa0159f"
server_key_version: "encryptionkey@0"
I20260925 14:45:05.169552 25273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:05.186347 25273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:05.188017 25273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:05.188162 25273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:05.191816 25273 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42029
I20260925 14:45:05.191834 25343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42029 every 8 connection(s)
I20260925 14:45:05.192289 25273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:05.192884 25344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:05.194929 25344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d: Bootstrap starting.
I20260925 14:45:05.195489 25344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:05.195817 25344 log.cc:824] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:05.196449 25344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d: No bootstrap required, opened a new log
I20260925 14:45:05.197960 25344 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } }
I20260925 14:45:05.198082 25344 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:05.198104 25344 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab32a1f91df24e94b33bcdc6a8a2ca1d, State: Initialized, Role: FOLLOWER
I20260925 14:45:05.198262 25344 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } }
I20260925 14:45:05.198329 25344 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:05.198350 25344 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:05.198414 25344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:05.199287 25344 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } }
I20260925 14:45:05.199331 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25273
I20260925 14:45:05.199366 25344 leader_election.cc:302] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab32a1f91df24e94b33bcdc6a8a2ca1d; no voters:
I20260925 14:45:05.199443 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:05.199537 25344 leader_election.cc:288] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:05.199576 25347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:05.199627 22327 external_mini_cluster.cc:1428] Setting key 9e897d5752da6ea52797dcb3139cc6f9
I20260925 14:45:05.199718 25347 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [term 1 LEADER]: Becoming Leader. State: Replica: ab32a1f91df24e94b33bcdc6a8a2ca1d, State: Running, Role: LEADER
I20260925 14:45:05.199833 25344 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:05.200121 25347 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } }
I20260925 14:45:05.200493 25348 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } } }
I20260925 14:45:05.200501 25349 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab32a1f91df24e94b33bcdc6a8a2ca1d. Latest consensus state: current_term: 1 leader_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab32a1f91df24e94b33bcdc6a8a2ca1d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42029 } } }
I20260925 14:45:05.200591 25348 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:05.200605 25349 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:05.200802 25358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:05.201349 25358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:05.202764 25358 catalog_manager.cc:1347] Generated new cluster ID: 88f219b50df64fb5bf8e1879b0fe4e0b
I20260925 14:45:05.202816 25358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:05.210708 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:05.200039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54592 (local address 127.21.205.254:42029)
0925 14:45:05.200276 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:05.200285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:05.200316 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:45:05.200426 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:05.200433 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:05.200483 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:05.201601 (+ 1118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:05.201701 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:05.203044 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:05.203762 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:05.204364 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:05.204751 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:05.207027 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:05.207031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:05.207031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:05.207041 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:05.208928 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:05.209270 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:05.209272 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:05.209273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:05.209370 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:05.209532 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:05.209533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:05.209533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:05.209811 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:05.210314 (+ 503us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:05.210376 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 14:45:05.210496 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":63,"threads_started":1}
I20260925 14:45:05.211365 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:45:05.225980 25358 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:05.226680 25358 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:05.226814 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:05.222742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54598 (local address 127.21.205.254:42029)
0925 14:45:05.222780 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:05.222782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:05.222888 (+ 106us) server_negotiation.cc:407] Connection header received
0925 14:45:05.222967 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:05.222968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:05.222976 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:05.223000 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:05.223398 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:05.224170 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:05.225166 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:05.225445 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:05.225485 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:05.226283 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:05.226284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:05.226285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:05.226471 (+ 186us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:05.226487 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:05.226494 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:05.226524 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:05.226651 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 14:45:05.226685 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:45:05.233606 25358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab32a1f91df24e94b33bcdc6a8a2ca1d: Generated new TSK 0
I20260925 14:45:05.233781 25358 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:45:05.238953 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25273
2026-09-25T14:45:05Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1286 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25377](info): set up 2 sockets
Sep 25 14:45:05 dist-test-slave-d7lw krb5kdc[25377](info): commencing operation
krb5kdc: starting...
W20260925 14:45:06.254333 22327 mini_kdc.cc:121] Time spent starting KDC: real 0.997s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:06.361821 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:41359
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:06.440101 25390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:06.440284 25390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:06.440304 25390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:06.441368 25390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:06.441409 25390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:06.441423 25390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:06.441435 25390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:06.441447 25390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:06.442988 25390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:41359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:06.443166 25390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:06.443324 25390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:06.447857 25396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.447909 25398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.447924 25390 server_base.cc:1034] running on GCE node
W20260925 14:45:06.447865 25395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.448266 25390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:06.448485 25390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:06.449653 25390 hybrid_clock.cc:648] HybridClock initialized: now 1790347506449642 us; error 24 us; skew 500 ppm
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:06.451637 25390 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:06.452062 25390 webserver.cc:460] Webserver started at http://127.21.205.254:45221/ using document root <none> and password file <none>
I20260925 14:45:06.452273 25390 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:06.452334 25390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:06.452487 25390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:06.453459 25390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "b0cb240f165e416a956bd2f99d9f8587"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "436e41cec1ad3898b2c21fa6e2414749"
server_key_iv: "5c9e2da2ff3ad932623ffada7af40c41"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.453797 25390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "b0cb240f165e416a956bd2f99d9f8587"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "436e41cec1ad3898b2c21fa6e2414749"
server_key_iv: "5c9e2da2ff3ad932623ffada7af40c41"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.455039 25390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:06.455725 25405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.455899 25390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:06.455994 25390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "b0cb240f165e416a956bd2f99d9f8587"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "436e41cec1ad3898b2c21fa6e2414749"
server_key_iv: "5c9e2da2ff3ad932623ffada7af40c41"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.456075 25390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:06.485054 25390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:06.486670 25390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:06.486853 25390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:06.490688 25390 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:41359
I20260925 14:45:06.490715 25458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:41359 every 8 connection(s)
I20260925 14:45:06.491024 25390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:06.492055 25459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:06.494107 25459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587: Bootstrap starting.
I20260925 14:45:06.494602 25459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:06.494837 25459 log.cc:824] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:06.495443 25459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587: No bootstrap required, opened a new log
I20260925 14:45:06.495808 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25390
I20260925 14:45:06.495916 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:06.496088 22327 external_mini_cluster.cc:1428] Setting key 69446be4eb8712b298e8358cc86b6d63
I20260925 14:45:06.497499 25459 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } }
I20260925 14:45:06.497668 25459 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:06.497746 25459 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0cb240f165e416a956bd2f99d9f8587, State: Initialized, Role: FOLLOWER
I20260925 14:45:06.497907 25459 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } }
I20260925 14:45:06.498000 25459 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:06.498034 25459 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:06.498088 25459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:06.498971 25459 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } }
I20260925 14:45:06.499053 25459 leader_election.cc:302] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0cb240f165e416a956bd2f99d9f8587; no voters:
I20260925 14:45:06.499228 25459 leader_election.cc:288] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:06.499264 25464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:06.499429 25464 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [term 1 LEADER]: Becoming Leader. State: Replica: b0cb240f165e416a956bd2f99d9f8587, State: Running, Role: LEADER
I20260925 14:45:06.499467 25459 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:06.499574 25464 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } }
I20260925 14:45:06.500001 25465 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0cb240f165e416a956bd2f99d9f8587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } } }
I20260925 14:45:06.500075 25465 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:06.500298 25466 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0cb240f165e416a956bd2f99d9f8587. Latest consensus state: current_term: 1 leader_uuid: "b0cb240f165e416a956bd2f99d9f8587" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0cb240f165e416a956bd2f99d9f8587" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41359 } } }
I20260925 14:45:06.500391 25466 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:06.500862 25471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:06.501459 25471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:06.503126 25471 catalog_manager.cc:1347] Generated new cluster ID: 395510929cd04eb8985f1a255bbc95f7
I20260925 14:45:06.503175 25471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:06.508272 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.496540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.21.205.254:41359)
0925 14:45:06.496725 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:06.496732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:06.496757 (+ 25us) server_negotiation.cc:407] Connection header received
0925 14:45:06.497488 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:06.497494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:06.497536 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:06.497641 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:06.498429 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.499483 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.500590 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.501333 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.501489 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.504258 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:06.504262 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:06.504262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:06.504273 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:06.506130 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.506610 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.506612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.506613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.506732 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.506961 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.506962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.506962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.507184 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:06.507977 (+ 793us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:06.508045 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 14:45:06.508146 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260925 14:45:06.516064 25471 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:06.516602 25471 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:06.519317 25471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0cb240f165e416a956bd2f99d9f8587: Generated new TSK 0
I20260925 14:45:06.519481 25471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:06.548943 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:06.647624 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:06.647842 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:06.647889 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:06.648967 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:06.649029 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:06.649076 25486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:06.650413 25486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:06.650674 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:06.650880 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:06.655608 25494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.655613 25491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.655903 25492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.656265 25486 server_base.cc:1034] running on GCE node
I20260925 14:45:06.656456 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:06.656702 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:06.657869 25486 hybrid_clock.cc:648] HybridClock initialized: now 1790347506657850 us; error 31 us; skew 500 ppm
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:06.660737 25486 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:06.661163 25486 webserver.cc:460] Webserver started at http://127.21.205.193:38721/ using document root <none> and password file <none>
I20260925 14:45:06.661368 25486 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:06.661429 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:06.661576 25486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:06.662511 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "84302cdc375049e3b2f77446e3077bc6"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "192446ba9d57e6c8b362bf50f6a62e0c"
server_key_iv: "e31581626bbc79eccafc47653ee77485"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.662847 25486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "84302cdc375049e3b2f77446e3077bc6"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "192446ba9d57e6c8b362bf50f6a62e0c"
server_key_iv: "e31581626bbc79eccafc47653ee77485"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.664335 25486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:06.665102 25501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.665267 25486 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:06.665345 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "84302cdc375049e3b2f77446e3077bc6"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "192446ba9d57e6c8b362bf50f6a62e0c"
server_key_iv: "e31581626bbc79eccafc47653ee77485"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.665395 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:06.672276 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:06.674477 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:06.674630 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:06.674836 25486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:06.675127 25486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:06.675175 25486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.675213 25486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:06.675239 25486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.680737 25486 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:41251
I20260925 14:45:06.680783 25618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:41251 every 8 connection(s)
I20260925 14:45:06.681057 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:45:06.683773 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25486
I20260925 14:45:06.683892 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:06.684106 22327 external_mini_cluster.cc:1428] Setting key 330e6c90b77dcce29948957adc8c0426
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:06.690099 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.681627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:47177 (local address 127.21.205.254:41359)
0925 14:45:06.681694 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:06.681696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:06.681908 (+ 212us) server_negotiation.cc:407] Connection header received
0925 14:45:06.682359 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:06.682360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:06.682372 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:06.682417 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:06.683239 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.683851 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.684873 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.685210 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.685298 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.687417 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:06.687419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:06.687419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:06.687428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:06.688754 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.689160 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.689161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.689161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.689216 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.689483 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.689484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.689484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.689603 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:06.689673 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:06.689924 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 14:45:06.690001 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:45:06.690264 25621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.681591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41359 (local address 127.21.205.193:47177)
0925 14:45:06.681765 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:06.681809 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:06.682287 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:06.682462 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:06.682465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:06.682992 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:06.683168 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.683172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.683940 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.683941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.684798 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.684801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.685311 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.685312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.685405 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.685842 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:06.685849 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:06.687328 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:06.688846 (+ 1518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.688853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.688854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.689054 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.689280 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.689281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.689281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.689388 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.689666 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:06.689673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:06.689801 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 14:45:06.689904 (+ 103us) client_negotiation.cc:240] Negotiation successful
0925 14:45:06.690044 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":90,"threads_started":1}
I20260925 14:45:06.690825 25619 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41359
I20260925 14:45:06.690918 25619 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:06.691088 25619 heartbeater.cc:511] Master 127.21.205.254:41359 requested a full tablet report, sending...
I20260925 14:45:06.691594 25423 ts_manager.cc:194] Registered new tserver with Master: 84302cdc375049e3b2f77446e3077bc6 (127.21.205.193:41251)
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
I20260925 14:45:06.692288 25423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:47177
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:06.707590 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:06.786000 25624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:06.786190 25624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:06.786211 25624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:06.787248 25624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:06.787279 25624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:06.787303 25624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:06.788620 25624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:06.788781 25624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:06.788926 25624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:06.793524 25629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.793529 25630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.793533 25632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.793815 25624 server_base.cc:1034] running on GCE node
I20260925 14:45:06.794011 25624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:06.794267 25624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:06.795420 25624 hybrid_clock.cc:648] HybridClock initialized: now 1790347506795411 us; error 38 us; skew 500 ppm
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:06.797832 25624 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:06.798215 25624 webserver.cc:460] Webserver started at http://127.21.205.194:44707/ using document root <none> and password file <none>
I20260925 14:45:06.798419 25624 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:06.798481 25624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:06.798630 25624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:06.799697 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "7301d3cd76a44ce1ad423fc39198a8c3"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "2a6059a186c889d9752f532dd2906d9f"
server_key_iv: "44f0a93f506bd1b71d3e523cf7edff0d"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.800129 25624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "7301d3cd76a44ce1ad423fc39198a8c3"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "2a6059a186c889d9752f532dd2906d9f"
server_key_iv: "44f0a93f506bd1b71d3e523cf7edff0d"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.801465 25624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:06.802186 25639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.802366 25624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:06.802456 25624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "7301d3cd76a44ce1ad423fc39198a8c3"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "2a6059a186c889d9752f532dd2906d9f"
server_key_iv: "44f0a93f506bd1b71d3e523cf7edff0d"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.802538 25624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:06.809705 25624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:06.811930 25624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:06.812075 25624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:06.812269 25624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:06.812582 25624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:06.812633 25624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.812674 25624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:06.812701 25624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.818154 25624 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:41037
I20260925 14:45:06.818225 25754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:41037 every 8 connection(s)
I20260925 14:45:06.818533 25624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:06.821643 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25624
I20260925 14:45:06.821736 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:06.821961 22327 external_mini_cluster.cc:1428] Setting key 004a738bace2a3f35f057907f8ba47b5
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:06.829082 25757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.819190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41359 (local address 127.21.205.194:37239)
0925 14:45:06.819322 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:06.819358 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:06.819761 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:06.820004 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:06.820007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:06.820767 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:06.821072 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.821078 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.822015 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.822016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.823281 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.823284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.823821 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.823822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.823940 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.824432 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:06.824440 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:06.825815 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:06.827728 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.827734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.827735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.828023 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.828243 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.828244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.828244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.828334 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.828627 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:06.828631 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:06.828729 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 14:45:06.828833 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 14:45:06.828934 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":73,"threads_started":1}
I20260925 14:45:06.829083 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.819217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:37239 (local address 127.21.205.254:41359)
0925 14:45:06.819258 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:06.819261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:06.819432 (+ 171us) server_negotiation.cc:407] Connection header received
0925 14:45:06.819891 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:06.819892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:06.819902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:06.819948 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:06.821153 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.821905 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.823386 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.823646 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.823704 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.825898 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:06.825900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:06.825900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:06.825908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:06.827638 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.828114 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.828115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.828116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.828177 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.828420 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.828421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.828421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.828567 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:06.828633 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:06.828897 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 14:45:06.828985 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:45:06.829627 25755 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41359
I20260925 14:45:06.829705 25755 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:06.829860 25755 heartbeater.cc:511] Master 127.21.205.254:41359 requested a full tablet report, sending...
I20260925 14:45:06.830227 25423 ts_manager.cc:194] Registered new tserver with Master: 7301d3cd76a44ce1ad423fc39198a8c3 (127.21.205.194:41037)
I20260925 14:45:06.830778 25423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:37239
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:06.846489 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:06.923820 25760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:06.923996 25760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:06.924018 25760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:06.925184 25760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:06.925225 25760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:06.925278 25760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:06.926676 25760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:06.926909 25760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:06.927119 25760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:06.932060 25766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:06.932052 25765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.932215 25760 server_base.cc:1034] running on GCE node
W20260925 14:45:06.932052 25768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:06.932555 25760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:06.932809 25760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:06.933987 25760 hybrid_clock.cc:648] HybridClock initialized: now 1790347506933975 us; error 53 us; skew 500 ppm
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:06.936359 25760 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:06.936714 25760 webserver.cc:460] Webserver started at http://127.21.205.195:45971/ using document root <none> and password file <none>
I20260925 14:45:06.936888 25760 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:06.936924 25760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:06.937005 25760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:06.937875 25760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "fa4b369307e34d3c9a5680f8d6175d16"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "0e9e98c2ad97354f14ffc56da25a87fd"
server_key_iv: "e04e1a4eeb0383b3cb30a667c219c144"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.938153 25760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "fa4b369307e34d3c9a5680f8d6175d16"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "0e9e98c2ad97354f14ffc56da25a87fd"
server_key_iv: "e04e1a4eeb0383b3cb30a667c219c144"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.939316 25760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 14:45:06.939976 25775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.940155 25760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.940231 25760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "fa4b369307e34d3c9a5680f8d6175d16"
format_stamp: "Formatted at 2026-09-25 14:45:06 on dist-test-slave-d7lw"
server_key: "0e9e98c2ad97354f14ffc56da25a87fd"
server_key_iv: "e04e1a4eeb0383b3cb30a667c219c144"
server_key_version: "encryptionkey@0"
I20260925 14:45:06.940325 25760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:06.964994 25760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:06.967276 25760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:06.967450 25760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:06.967671 25760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:06.967985 25760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:06.968019 25760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.968098 25760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:06.968122 25760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:06.973800 25760 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:44579
I20260925 14:45:06.973853 25890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:44579 every 8 connection(s)
I20260925 14:45:06.974148 25760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:06.980707 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25760
I20260925 14:45:06.980831 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:06.981017 22327 external_mini_cluster.cc:1428] Setting key 24b4b2e887bd1f653ed5ef478870add7
I20260925 14:45:06.983115 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.974793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:37617 (local address 127.21.205.254:41359)
0925 14:45:06.974868 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:06.974871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:06.975107 (+ 236us) server_negotiation.cc:407] Connection header received
0925 14:45:06.975541 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:06.975542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:06.975552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:06.975621 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:06.976563 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.977037 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.977935 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.978100 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.978144 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.980083 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:06.980084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:06.980085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:06.980091 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:06.981685 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.982133 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.982134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.982135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.982189 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:06.982439 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:06.982440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:06.982441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:06.982571 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:06.982655 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:06.982934 (+ 279us) server_negotiation.cc:299] Negotiation successful
0925 14:45:06.983015 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:45:06.983282 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:06.974792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41359 (local address 127.21.205.195:37617)
0925 14:45:06.974987 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:06.975021 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:06.975463 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:06.975691 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:06.975695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:06.976294 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:06.976487 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.976491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.977111 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.977112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.977866 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:06.977868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:06.978163 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:06.978164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:06.978258 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:06.978668 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:06.978676 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:06.980004 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:06.981767 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.981773 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.981775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.982051 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.982257 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:06.982258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:06.982259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:06.982371 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:06.982657 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:06.982663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:06.982789 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 14:45:06.982913 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 14:45:06.983057 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":103,"threads_started":1}
I20260925 14:45:06.983651 25891 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41359
I20260925 14:45:06.983740 25891 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:06.983922 25891 heartbeater.cc:511] Master 127.21.205.254:41359 requested a full tablet report, sending...
I20260925 14:45:06.984292 25423 ts_manager.cc:194] Registered new tserver with Master: fa4b369307e34d3c9a5680f8d6175d16 (127.21.205.195:44579)
I20260925 14:45:06.984591 25423 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:37617
I20260925 14:45:06.992731 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 14:45:06 dist-test-slave-d7lw krb5kdc[25377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 14:45:07 dist-test-slave-d7lw krb5kdc[25377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347506, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:07.014079 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:07.007799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.21.205.254:41359)
0925 14:45:07.007839 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:07.007841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:07.007980 (+ 139us) server_negotiation.cc:407] Connection header received
0925 14:45:07.008065 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:07.008066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:07.008075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:07.008136 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:07.008638 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:07.009083 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:07.009792 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:07.009975 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:07.010049 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:07.012105 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:07.012107 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:07.012107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:07.012112 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:07.013217 (+ 1105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:07.013510 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:07.013511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:07.013511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:07.013564 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:07.013707 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:07.013708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:07.013708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:07.013818 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:07.013885 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:07.013969 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 14:45:07.014022 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 14:45:07.014279 25423 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33422
I20260925 14:45:07.014671 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25486
I20260925 14:45:07.019550 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25624
I20260925 14:45:07.025308 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25760
I20260925 14:45:07.029659 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25390
2026-09-25T14:45:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1792 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:07 dist-test-slave-d7lw krb5kdc[25903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:07 dist-test-slave-d7lw krb5kdc[25903](info): set up 2 sockets
Sep 25 14:45:07 dist-test-slave-d7lw krb5kdc[25903](info): commencing operation
krb5kdc: starting...
W20260925 14:45:08.054672 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:08.164359 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:36971
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:36971
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:08.241602 25916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:08.241777 25916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:08.241798 25916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:08.242841 25916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:08.242871 25916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:08.242885 25916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:08.242898 25916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:08.242910 25916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:08.244314 25916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:36971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:36971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:08.244505 25916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:08.244656 25916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:08.249343 25921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.249343 25924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.249567 25916 server_base.cc:1034] running on GCE node
W20260925 14:45:08.249668 25922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.249940 25916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:08.250221 25916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:08.251392 25916 hybrid_clock.cc:648] HybridClock initialized: now 1790347508251362 us; error 42 us; skew 500 ppm
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:08.259727 25916 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:08.260262 25916 webserver.cc:460] Webserver started at http://127.21.205.254:33317/ using document root <none> and password file <none>
I20260925 14:45:08.260521 25916 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:08.260607 25916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:08.260740 25916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:08.261921 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "8797aa3cba8f06ccaf45c0a3885b552f"
server_key_iv: "56dd1c6f8abc366ec7f632f932a8478d"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.262228 25916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "8797aa3cba8f06ccaf45c0a3885b552f"
server_key_iv: "56dd1c6f8abc366ec7f632f932a8478d"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.263413 25916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:45:08.264089 25931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.264266 25916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:08.264359 25916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "8797aa3cba8f06ccaf45c0a3885b552f"
server_key_iv: "56dd1c6f8abc366ec7f632f932a8478d"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.264446 25916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:08.271389 25916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:08.273159 25916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:08.273327 25916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:08.277200 25916 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:36971
I20260925 14:45:08.277230 25984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:36971 every 8 connection(s)
I20260925 14:45:08.277745 25916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:08.277940 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 25916
I20260925 14:45:08.278061 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:08.278237 22327 external_mini_cluster.cc:1428] Setting key adbd801690a52ce6856fea89a2717f05
I20260925 14:45:08.278578 25985 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:08.281782 25985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0: Bootstrap starting.
I20260925 14:45:08.282400 25985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:08.282698 25985 log.cc:824] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:08.283507 25985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0: No bootstrap required, opened a new log
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:08.285387 25985 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } }
I20260925 14:45:08.285540 25985 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:08.285585 25985 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e1ee16fc74c4e63a6d7e7e312b0aec0, State: Initialized, Role: FOLLOWER
I20260925 14:45:08.285742 25985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } }
I20260925 14:45:08.285830 25985 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:08.285867 25985 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:08.285926 25985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:08.286531 25985 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } }
I20260925 14:45:08.286614 25985 leader_election.cc:302] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e1ee16fc74c4e63a6d7e7e312b0aec0; no voters:
I20260925 14:45:08.286824 25985 leader_election.cc:288] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:08.286852 25990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:08.286967 25990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [term 1 LEADER]: Becoming Leader. State: Replica: 1e1ee16fc74c4e63a6d7e7e312b0aec0, State: Running, Role: LEADER
I20260925 14:45:08.287101 25985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:08.287132 25990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } }
I20260925 14:45:08.287496 25990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e1ee16fc74c4e63a6d7e7e312b0aec0. Latest consensus state: current_term: 1 leader_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } } }
I20260925 14:45:08.287592 25990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:08.287750 25993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e1ee16fc74c4e63a6d7e7e312b0aec0" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36971 } } }
I20260925 14:45:08.288064 25993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:08.288061 26000 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:08.288689 26000 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:08.289134 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.278790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43852 (local address 127.21.205.254:36971)
0925 14:45:08.278971 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.278980 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.279011 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:45:08.279661 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.279669 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.279736 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.279904 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.280443 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.281766 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.282529 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.283011 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.283123 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.285391 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.285394 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.285395 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.285404 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.287272 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.287699 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.287702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.287702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.287821 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.288049 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.288050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.288050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.288333 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.288477 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.288776 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.288913 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
I20260925 14:45:08.292708 26000 catalog_manager.cc:1347] Generated new cluster ID: c9d977a0d4cf47ca9401d5e38aa8c64f
I20260925 14:45:08.292760 26000 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:08.319202 26000 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:08.319973 26000 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:08.329487 26000 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e1ee16fc74c4e63a6d7e7e312b0aec0: Generated new TSK 0
I20260925 14:45:08.329691 26000 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:08.355031 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:08.433333 26012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:08.433511 26012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:08.433532 26012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:08.434643 26012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:08.434680 26012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:08.434731 26012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:08.436228 26012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:08.436463 26012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:08.436679 26012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:08.441401 26018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.441396 26017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.441393 26020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.441645 26012 server_base.cc:1034] running on GCE node
I20260925 14:45:08.441862 26012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:08.442085 26012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:08.443243 26012 hybrid_clock.cc:648] HybridClock initialized: now 1790347508443230 us; error 27 us; skew 500 ppm
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:08.445761 26012 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:08.446141 26012 webserver.cc:460] Webserver started at http://127.21.205.193:45019/ using document root <none> and password file <none>
I20260925 14:45:08.446369 26012 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:08.446435 26012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:08.446585 26012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:08.447613 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "3ca26713b3c3452ebf3ee9319c3960f2"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "df1eeda01583bf4e68961dd8da352cf9"
server_key_iv: "1585b5ffd9363c878c958ac5e4321d49"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.447999 26012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "3ca26713b3c3452ebf3ee9319c3960f2"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "df1eeda01583bf4e68961dd8da352cf9"
server_key_iv: "1585b5ffd9363c878c958ac5e4321d49"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.449311 26012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:08.450013 26027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.450170 26012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:08.450265 26012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "3ca26713b3c3452ebf3ee9319c3960f2"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "df1eeda01583bf4e68961dd8da352cf9"
server_key_iv: "1585b5ffd9363c878c958ac5e4321d49"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.450340 26012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:08.461500 26012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:08.463652 26012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:08.463874 26012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:08.464083 26012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:08.464371 26012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:08.464422 26012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.464459 26012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:08.464484 26012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.470515 26012 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:41087
I20260925 14:45:08.470579 26141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:41087 every 8 connection(s)
I20260925 14:45:08.470865 26012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:08.478822 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.471532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56595 (local address 127.21.205.254:36971)
0925 14:45:08.471618 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.471620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.471800 (+ 180us) server_negotiation.cc:407] Connection header received
0925 14:45:08.472663 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.472664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.472674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.472725 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.473243 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.473725 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.474640 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.474825 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.474872 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.476714 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.476715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.476716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.476722 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.477783 (+ 1061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.478102 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.478103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.478103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.478148 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.478326 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.478326 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.478327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.478437 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.478485 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.478696 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.478763 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 14:45:08.478979 26144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.471497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:36971 (local address 127.21.205.193:56595)
0925 14:45:08.471663 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:08.471695 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:08.472576 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:08.472773 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:08.472775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:08.472997 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:08.473172 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.473175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.473797 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.473798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.474570 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.474572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.474892 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.474893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.474983 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.475343 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:08.475351 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:08.476636 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:08.477846 (+ 1210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.477852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.477853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.478043 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.478201 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.478201 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.478202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.478277 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.478492 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:08.478497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:08.478595 (+ 98us) client_negotiation.cc:769] Sending connection context
0925 14:45:08.478670 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 14:45:08.478783 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260925 14:45:08.479526 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26012
I20260925 14:45:08.479596 26142 heartbeater.cc:347] Connected to a master server at 127.21.205.254:36971
I20260925 14:45:08.479645 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:08.479691 26142 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:08.479859 22327 external_mini_cluster.cc:1428] Setting key f534c78a3fa9956442bc37f2f01f06d3
I20260925 14:45:08.479875 26142 heartbeater.cc:511] Master 127.21.205.254:36971 requested a full tablet report, sending...
I20260925 14:45:08.480408 25948 ts_manager.cc:194] Registered new tserver with Master: 3ca26713b3c3452ebf3ee9319c3960f2 (127.21.205.193:41087)
I20260925 14:45:08.481240 25948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:56595
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:08.502265 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:08.580960 26147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:08.581199 26147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:08.581236 26147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:08.582371 26147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:08.582437 26147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:08.582482 26147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:08.583962 26147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:08.584230 26147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:08.584435 26147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:08.589359 26152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.589471 26147 server_base.cc:1034] running on GCE node
W20260925 14:45:08.589385 26153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.589421 26155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.589811 26147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:08.590088 26147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:08.591223 26147 hybrid_clock.cc:648] HybridClock initialized: now 1790347508591210 us; error 27 us; skew 500 ppm
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:08.593614 26147 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:08.594054 26147 webserver.cc:460] Webserver started at http://127.21.205.194:43647/ using document root <none> and password file <none>
I20260925 14:45:08.594270 26147 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:08.594337 26147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:08.594499 26147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:08.595422 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "d0da11a5f6b44adc9f3803299bd0f3b8"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "b09da00d4e62ccb894d150d853791159"
server_key_iv: "aeb5c0f19cb9e4ff764acc80a023247c"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.595764 26147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "d0da11a5f6b44adc9f3803299bd0f3b8"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "b09da00d4e62ccb894d150d853791159"
server_key_iv: "aeb5c0f19cb9e4ff764acc80a023247c"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.597056 26147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:08.597728 26162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.597895 26147 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:08.597991 26147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "d0da11a5f6b44adc9f3803299bd0f3b8"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "b09da00d4e62ccb894d150d853791159"
server_key_iv: "aeb5c0f19cb9e4ff764acc80a023247c"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.598069 26147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:08.626452 26147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:08.628608 26147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:08.628782 26147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:08.629010 26147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:08.629314 26147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:08.629366 26147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.629423 26147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:08.629451 26147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.635069 26147 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:40675
I20260925 14:45:08.635119 26279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:40675 every 8 connection(s)
I20260925 14:45:08.635396 26147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:08.636976 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26147
I20260925 14:45:08.637096 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:08.637311 22327 external_mini_cluster.cc:1428] Setting key 9ab78a276448e692befb7af279533b73
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:08.645735 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.636095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:41867 (local address 127.21.205.254:36971)
0925 14:45:08.636138 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.636140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.636354 (+ 214us) server_negotiation.cc:407] Connection header received
0925 14:45:08.637554 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.637556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.637569 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.637612 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.638374 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.638839 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.640148 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.640370 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.640454 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.642827 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.642828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.642828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.642835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.644274 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.644775 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.644776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.644776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.644826 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.645099 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.645100 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.645100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.645201 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.645249 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.645605 (+ 356us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.645665 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
I20260925 14:45:08.646131 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.636083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:36971 (local address 127.21.205.194:41867)
0925 14:45:08.636246 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:08.636277 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:08.637453 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:08.637686 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:08.637690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:08.637945 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:08.638282 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.638288 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.638932 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.638934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.640048 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.640051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.640459 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.640460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.640590 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.641100 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:08.641108 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:08.642729 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:08.644355 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.644362 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.644364 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.644696 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.644900 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.644902 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.644902 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.645035 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.645269 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:08.645275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:08.645446 (+ 171us) client_negotiation.cc:769] Sending connection context
0925 14:45:08.645579 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 14:45:08.645763 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":76,"threads_started":1}
I20260925 14:45:08.646675 26280 heartbeater.cc:347] Connected to a master server at 127.21.205.254:36971
I20260925 14:45:08.646777 26280 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:08.646986 26280 heartbeater.cc:511] Master 127.21.205.254:36971 requested a full tablet report, sending...
I20260925 14:45:08.647444 25948 ts_manager.cc:194] Registered new tserver with Master: d0da11a5f6b44adc9f3803299bd0f3b8 (127.21.205.194:40675)
I20260925 14:45:08.647897 25948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:41867
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:08.662149 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:08.740538 26285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:08.740707 26285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:08.740728 26285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:08.741907 26285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:08.741946 26285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:08.741971 26285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:08.743530 26285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:36971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:08.743753 26285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:08.744009 26285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:08.748754 26293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.748742 26291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:08.748742 26290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:08.749001 26285 server_base.cc:1034] running on GCE node
I20260925 14:45:08.749212 26285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:08.749451 26285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:08.750602 26285 hybrid_clock.cc:648] HybridClock initialized: now 1790347508750583 us; error 31 us; skew 500 ppm
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:08.753041 26285 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:08.753463 26285 webserver.cc:460] Webserver started at http://127.21.205.195:41195/ using document root <none> and password file <none>
I20260925 14:45:08.753695 26285 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:08.753760 26285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:08.753909 26285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:08.754859 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "e56e9396012b447fad1938c5f2efa336"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "ba59782560e09a3a981ceeef3ee3c8c4"
server_key_iv: "1431f917556400db25f1e07214fa0087"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.755184 26285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "e56e9396012b447fad1938c5f2efa336"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "ba59782560e09a3a981ceeef3ee3c8c4"
server_key_iv: "1431f917556400db25f1e07214fa0087"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.756544 26285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:08.757264 26300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.757452 26285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.757539 26285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "e56e9396012b447fad1938c5f2efa336"
format_stamp: "Formatted at 2026-09-25 14:45:08 on dist-test-slave-d7lw"
server_key: "ba59782560e09a3a981ceeef3ee3c8c4"
server_key_iv: "1431f917556400db25f1e07214fa0087"
server_key_version: "encryptionkey@0"
I20260925 14:45:08.757612 26285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:08.790581 26285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:08.792469 26285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:08.792619 26285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:08.792837 26285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:08.793141 26285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:08.793195 26285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.793256 26285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:08.793284 26285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:08.798772 26285 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:37027
I20260925 14:45:08.798885 26416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:37027 every 8 connection(s)
I20260925 14:45:08.799161 26285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:08.806653 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26285
I20260925 14:45:08.806749 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:08.806945 22327 external_mini_cluster.cc:1428] Setting key 9073520f4acab010b236c4c514c9e2ee
I20260925 14:45:08.808168 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.799894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:48303 (local address 127.21.205.254:36971)
0925 14:45:08.799944 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.799946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.800113 (+ 167us) server_negotiation.cc:407] Connection header received
0925 14:45:08.801079 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.801080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.801090 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.801139 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.801728 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.802218 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.803165 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.803351 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.803399 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.805487 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.805488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.805489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.805495 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.806667 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.807146 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.807147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.807147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.807207 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.807469 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.807470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.807470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.807582 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.807651 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.808006 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.808082 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:45:08.808374 26419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.799864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:36971 (local address 127.21.205.195:48303)
0925 14:45:08.800009 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:08.800046 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:08.800992 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:08.801197 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:08.801200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:08.801423 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:08.801649 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.801654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.802297 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.802297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.803094 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:08.803096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.803419 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.803420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:08.803518 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.804067 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:08.804075 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:08.805380 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:08.806770 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.806777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.806779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.807071 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.807286 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:08.807287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:08.807287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:08.807396 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:08.807653 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:08.807659 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:08.807865 (+ 206us) client_negotiation.cc:769] Sending connection context
0925 14:45:08.807984 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 14:45:08.808128 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260925 14:45:08.808786 26417 heartbeater.cc:347] Connected to a master server at 127.21.205.254:36971
I20260925 14:45:08.808887 26417 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:08.809062 26417 heartbeater.cc:511] Master 127.21.205.254:36971 requested a full tablet report, sending...
I20260925 14:45:08.809397 25948 ts_manager.cc:194] Registered new tserver with Master: e56e9396012b447fad1938c5f2efa336 (127.21.205.195:37027)
I20260925 14:45:08.809758 25948 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:48303
I20260925 14:45:08.818603 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:45:08.841943 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.834531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.21.205.193:41087)
0925 14:45:08.834693 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.834698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.834710 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:08.834746 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.834747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.834761 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.834829 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.835348 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.836138 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.836813 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.837119 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.837171 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.839216 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.839225 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.839225 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.839231 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.840953 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.841253 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.841256 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.841256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.841309 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.841483 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.841484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.841484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.841614 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.841715 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.841775 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.841831 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[25903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:08.849325 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:08.843204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43856 (local address 127.21.205.254:36971)
0925 14:45:08.843275 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:08.843276 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:08.843284 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:08.843322 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:08.843323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:08.843331 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:08.843375 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:08.843841 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.844300 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.844922 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:08.845103 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:08.845148 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:08.847078 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:08.847078 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:08.847079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:08.847084 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:08.848423 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.848705 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.848707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.848707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.848765 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:08.848902 (+ 137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:08.848902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:08.848902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:08.849021 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:08.849087 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:08.849166 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 14:45:08.849219 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 14:45:08.849603 25948 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43856
I20260925 14:45:08.849960 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26012
I20260925 14:45:08.855650 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26147
I20260925 14:45:08.859710 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26285
I20260925 14:45:08.864070 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 25916
2026-09-25T14:45:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1834 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[26429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[26429](info): set up 2 sockets
Sep 25 14:45:08 dist-test-slave-d7lw krb5kdc[26429](info): commencing operation
krb5kdc: starting...
W20260925 14:45:09.886555 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:09 dist-test-slave-d7lw krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:09.998497 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34527
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:34527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:10.082252 26442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:10.082468 26442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:10.082502 26442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:10.084071 26442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:10.084126 26442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:10.084172 26442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:10.084195 26442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:10.084216 26442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:10.086314 26442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:34527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:10.086587 26442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:10.086844 26442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:10.091666 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.091682 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.091697 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.091953 26442 server_base.cc:1034] running on GCE node
I20260925 14:45:10.092204 26442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:10.092459 26442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:10.093629 26442 hybrid_clock.cc:648] HybridClock initialized: now 1790347510093609 us; error 34 us; skew 500 ppm
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:10.096024 26442 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:10.096344 26442 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:10.096410 26442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:10.096549 26442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:10.097496 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "ca607b6b60a04fcf98880e4076257c4a"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "1205cbd673c426e6935ff437e7561054"
server_key_iv: "40e22d9d91636610c2af763b7bcfbae7"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.097824 26442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "ca607b6b60a04fcf98880e4076257c4a"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "1205cbd673c426e6935ff437e7561054"
server_key_iv: "40e22d9d91636610c2af763b7bcfbae7"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.099012 26442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:10.099649 26456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.099843 26442 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:10.099932 26442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "ca607b6b60a04fcf98880e4076257c4a"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "1205cbd673c426e6935ff437e7561054"
server_key_iv: "40e22d9d91636610c2af763b7bcfbae7"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.100009 26442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:10.111920 26442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:10.114060 26442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:10.114208 26442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:10.118151 26442 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:34527
I20260925 14:45:10.118180 26509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:34527 every 8 connection(s)
I20260925 14:45:10.118427 26442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:10.119309 26510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:10.121450 26510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: Bootstrap starting.
I20260925 14:45:10.122002 26510 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:10.122339 26510 log.cc:824] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:10.122359 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26442
I20260925 14:45:10.122455 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:10.122653 22327 external_mini_cluster.cc:1428] Setting key 382fe1fc59ee0cccb975de1dcd7c3a7e
I20260925 14:45:10.123003 26510 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: No bootstrap required, opened a new log
I20260925 14:45:10.125327 26510 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } }
I20260925 14:45:10.125468 26510 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:10.125506 26510 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca607b6b60a04fcf98880e4076257c4a, State: Initialized, Role: FOLLOWER
I20260925 14:45:10.125658 26510 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } }
I20260925 14:45:10.125751 26510 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:10.125794 26510 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:10.125842 26510 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:10.126339 26510 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } }
I20260925 14:45:10.126466 26510 leader_election.cc:302] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca607b6b60a04fcf98880e4076257c4a; no voters:
I20260925 14:45:10.126642 26510 leader_election.cc:288] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:10.126677 26515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:10.126802 26515 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [term 1 LEADER]: Becoming Leader. State: Replica: ca607b6b60a04fcf98880e4076257c4a, State: Running, Role: LEADER
I20260925 14:45:10.126888 26510 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:10.126879 26515 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } }
I20260925 14:45:10.127228 26516 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca607b6b60a04fcf98880e4076257c4a. Latest consensus state: current_term: 1 leader_uuid: "ca607b6b60a04fcf98880e4076257c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } } }
I20260925 14:45:10.127386 26516 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:10.127214 26517 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca607b6b60a04fcf98880e4076257c4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca607b6b60a04fcf98880e4076257c4a" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34527 } } }
I20260925 14:45:10.127924 26517 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:10.127820 26524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:10.128466 26524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:10.130088 26524 catalog_manager.cc:1347] Generated new cluster ID: bc26d6f73dae404586cc77b1568be8ff
I20260925 14:45:10.130136 26524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347509, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:10.137730 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.123152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.21.205.254:34527)
0925 14:45:10.123330 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.123339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.123367 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:45:10.124266 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.124274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.124331 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.124487 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.125076 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.126114 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.127199 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.127821 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.127912 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.134330 (+ 6418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.134334 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.134334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:10.134345 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.136181 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.136572 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.136574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.136574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.136664 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.136877 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.136878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.136878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.137134 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.137262 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.137404 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.137521 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260925 14:45:10.144680 26524 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:10.145345 26524 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:10.156481 26524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: Generated new TSK 0
I20260925 14:45:10.156644 26524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:10.181802 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:10.260851 26536 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:10.261023 26536 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:10.261045 26536 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:10.262092 26536 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:10.262121 26536 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:10.262146 26536 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:10.263494 26536 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26536
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:10.263653 26536 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:10.263875 26536 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:10.268579 26545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.268579 26542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.268618 26543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.268884 26536 server_base.cc:1034] running on GCE node
I20260925 14:45:10.269078 26536 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:10.269253 26536 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:10.270398 26536 hybrid_clock.cc:648] HybridClock initialized: now 1790347510270384 us; error 27 us; skew 500 ppm
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:10.272933 26536 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:10.273245 26536 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:10.273289 26536 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:10.273375 26536 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:10.274303 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "326fbfa0d9d5419db4229486996ced25"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "77a7fbf477cd93daf6825081e2ce6112"
server_key_iv: "1b4c1352eee0715b2632bae0ec5bd4fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.274603 26536 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "326fbfa0d9d5419db4229486996ced25"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "77a7fbf477cd93daf6825081e2ce6112"
server_key_iv: "1b4c1352eee0715b2632bae0ec5bd4fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.275741 26536 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:10.276562 26551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.276746 26536 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:10.276847 26536 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "326fbfa0d9d5419db4229486996ced25"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "77a7fbf477cd93daf6825081e2ce6112"
server_key_iv: "1b4c1352eee0715b2632bae0ec5bd4fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.276925 26536 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:10.290756 26536 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:10.292958 26536 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:10.293118 26536 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:10.293362 26536 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:10.293661 26536 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:10.293713 26536 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.293756 26536 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:10.293780 26536 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.299219 26536 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:39147
I20260925 14:45:10.299289 26665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:39147 every 8 connection(s)
I20260925 14:45:10.299556 26536 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:10.306119 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26536
I20260925 14:45:10.306221 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:10.306427 22327 external_mini_cluster.cc:1428] Setting key 5d8dd1de5de7b9f0dca87aabc8e44b38
I20260925 14:45:10.308591 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.300268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:38145 (local address 127.21.205.254:34527)
0925 14:45:10.300321 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.300323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.300515 (+ 192us) server_negotiation.cc:407] Connection header received
0925 14:45:10.301038 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.301040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.301052 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.301102 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.301996 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.302531 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.303543 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.303759 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.303882 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.305754 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.305756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.305756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:10.305763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.307276 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.307669 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.307670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.307671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.307722 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.307976 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.307977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.307978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.308088 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.308174 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.308428 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.308507 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 14:45:10.308772 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.300238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34527 (local address 127.21.205.193:38145)
0925 14:45:10.300395 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.300435 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.300927 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.301175 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.301178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.301420 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:10.301903 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.301908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.302641 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.302642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.303457 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.303459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.303903 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.303904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.304024 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.304413 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:10.304420 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:10.305646 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:10.307365 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.307372 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.307373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.307576 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.307809 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.307810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.307810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.307909 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.308168 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:10.308174 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:10.308307 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.308409 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.308547 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260925 14:45:10.309159 26666 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34527
I20260925 14:45:10.309242 26666 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:10.309401 26666 heartbeater.cc:511] Master 127.21.205.254:34527 requested a full tablet report, sending...
I20260925 14:45:10.309883 26473 ts_manager.cc:194] Registered new tserver with Master: 326fbfa0d9d5419db4229486996ced25 (127.21.205.193:39147)
I20260925 14:45:10.310564 26473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:38145
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:10.328788 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:10.406590 26671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:10.406801 26671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:10.406838 26671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:10.407985 26671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:10.408056 26671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:10.408103 26671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:10.409567 26671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:10.409830 26671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:10.410050 26671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:10.414693 26679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.414752 26677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.414762 26676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.415024 26671 server_base.cc:1034] running on GCE node
I20260925 14:45:10.415238 26671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:10.415450 26671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:10.416586 26671 hybrid_clock.cc:648] HybridClock initialized: now 1790347510416574 us; error 30 us; skew 500 ppm
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:10.418888 26671 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:10.419237 26671 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:10.419306 26671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:10.419450 26671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:10.420470 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "8b78ba68b46f456eb136b90df07f034e"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "7e490fe4330fb98aeadd888c6a468dc5"
server_key_iv: "741519b66f9be0ad2ae4b35c288da3f4"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.420812 26671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "8b78ba68b46f456eb136b90df07f034e"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "7e490fe4330fb98aeadd888c6a468dc5"
server_key_iv: "741519b66f9be0ad2ae4b35c288da3f4"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.422020 26671 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:10.422734 26686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.422883 26671 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:10.422982 26671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "8b78ba68b46f456eb136b90df07f034e"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "7e490fe4330fb98aeadd888c6a468dc5"
server_key_iv: "741519b66f9be0ad2ae4b35c288da3f4"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.423053 26671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:10.458336 26671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:10.460022 26671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:10.460167 26671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:10.460359 26671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:10.460630 26671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:10.460683 26671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.460722 26671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:10.460749 26671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.466210 26671 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:40317
I20260925 14:45:10.466277 26799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:40317 every 8 connection(s)
I20260925 14:45:10.466538 26671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:10.473096 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26671
I20260925 14:45:10.473227 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:10.473417 22327 external_mini_cluster.cc:1428] Setting key 546325ce192593a0c0f7a2a6406ca7ef
I20260925 14:45:10.475903 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.467211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:54763 (local address 127.21.205.254:34527)
0925 14:45:10.467295 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.467298 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.467490 (+ 192us) server_negotiation.cc:407] Connection header received
0925 14:45:10.468018 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.468019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.468030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.468086 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.469059 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.469572 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.470517 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.470743 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.470828 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.472829 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.472831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.472831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:10.472840 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.474375 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.474866 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.474866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.474867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.474912 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.475210 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.475211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.475211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.475302 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.475350 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.475667 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.475755 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 14:45:10.476034 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.467187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34527 (local address 127.21.205.194:54763)
0925 14:45:10.467359 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.467406 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.467921 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.468150 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.468154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.468419 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:10.468931 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.468936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.469661 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.469662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.470425 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.470427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.470839 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.470840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.470970 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.471462 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:10.471470 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:10.472732 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:10.474470 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.474475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.474477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.474791 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.474993 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.474994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.474995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.475141 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.475364 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:10.475368 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:10.475522 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.475630 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.475827 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260925 14:45:10.476485 26800 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34527
I20260925 14:45:10.476572 26800 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:10.476713 26800 heartbeater.cc:511] Master 127.21.205.254:34527 requested a full tablet report, sending...
I20260925 14:45:10.477053 26473 ts_manager.cc:194] Registered new tserver with Master: 8b78ba68b46f456eb136b90df07f034e (127.21.205.194:40317)
I20260925 14:45:10.477547 26473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:54763
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:10.497993 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:10.575435 26805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:10.575628 26805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:10.575651 26805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:10.576786 26805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:10.576831 26805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:10.576859 26805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:10.578217 26805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:10.578402 26805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:10.578562 26805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:10.583419 26811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.583426 26810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.583425 26813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.583701 26805 server_base.cc:1034] running on GCE node
I20260925 14:45:10.583946 26805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:10.584198 26805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:10.585361 26805 hybrid_clock.cc:648] HybridClock initialized: now 1790347510585342 us; error 45 us; skew 500 ppm
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:10.587971 26805 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:10.588354 26805 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:10.588420 26805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:10.588594 26805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:10.589545 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "2bd145f84882455daaa023618dbb9af2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "415327b953fce635b5326595430c5343"
server_key_iv: "34641e5f0859784eb816b4ca35a982e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.589947 26805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "2bd145f84882455daaa023618dbb9af2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "415327b953fce635b5326595430c5343"
server_key_iv: "34641e5f0859784eb816b4ca35a982e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.591213 26805 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:45:10.591917 26819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.592087 26805 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.592182 26805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "2bd145f84882455daaa023618dbb9af2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "415327b953fce635b5326595430c5343"
server_key_iv: "34641e5f0859784eb816b4ca35a982e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.592264 26805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:10.602875 26805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:10.604758 26805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:10.604914 26805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:10.605120 26805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:10.605434 26805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:10.605501 26805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.605543 26805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:10.605573 26805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.611701 26805 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:35571
I20260925 14:45:10.611729 26933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:35571 every 8 connection(s)
I20260925 14:45:10.612037 26805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:10 dist-test-slave-d7lw krb5kdc[26429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347510, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:10.620702 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.612680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:53971 (local address 127.21.205.254:34527)
0925 14:45:10.612736 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.612738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.612912 (+ 174us) server_negotiation.cc:407] Connection header received
0925 14:45:10.613413 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.613414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.613427 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.613477 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.614335 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.614829 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.615840 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.616053 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.616142 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.618122 (+ 1980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.618123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.618124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:10.618138 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.619336 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.619730 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.619732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.619732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.619850 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.620091 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.620092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.620092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.620218 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.620284 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.620540 (+ 256us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.620618 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 14:45:10.620862 26936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.612650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34527 (local address 127.21.205.195:53971)
0925 14:45:10.612797 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.612837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.613305 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.613538 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.613542 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.613786 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:10.614232 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.614236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.614912 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.614913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.615694 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.615695 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.616143 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.616144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.616266 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.616698 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:10.616706 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:10.618021 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:10.619429 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.619435 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.619437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.619656 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.619912 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:10.619913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:10.619914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:10.620013 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:10.620286 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:10.620291 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:10.620413 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.620507 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.620654 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260925 14:45:10.621320 26934 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34527
I20260925 14:45:10.621388 26934 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:10.621558 26934 heartbeater.cc:511] Master 127.21.205.254:34527 requested a full tablet report, sending...
I20260925 14:45:10.621928 26473 ts_manager.cc:194] Registered new tserver with Master: 2bd145f84882455daaa023618dbb9af2 (127.21.205.195:35571)
I20260925 14:45:10.622125 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26805
I20260925 14:45:10.622221 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:10.622411 22327 external_mini_cluster.cc:1428] Setting key 6b790d9379d6cc1f9f184fbf69267969
I20260925 14:45:10.622453 26473 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:53971
I20260925 14:45:10.623229 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:45:10.632393 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.624545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53898 (local address 127.21.205.254:34527)
0925 14:45:10.624605 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.624607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.624713 (+ 106us) server_negotiation.cc:407] Connection header received
0925 14:45:10.624783 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.624784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.624792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.624838 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.625487 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.626300 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.627541 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.627929 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.628045 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.629682 (+ 1637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.629683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.629684 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:10.629690 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.631252 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.631549 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.631550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.631550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.631600 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:10.631805 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:10.631806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:10.631807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:10.631971 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.632072 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.632242 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.632312 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 14:45:10.634997 26473 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:10.637106 26473 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:10.643143 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48664 (local address 127.21.205.195:35571)
0925 14:45:10.639760 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.639765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.639773 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:10.639824 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.639827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.639841 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.639931 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:10.640133 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.641099 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642011 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642718 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642796 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.642867 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.643015 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.643067 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":64,"threads_started":1}
I20260925 14:45:10.643139 26944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:35571 (local address 127.0.0.1:48664)
0925 14:45:10.639474 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.639482 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.639513 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.640003 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.640004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.640006 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:10.640056 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.640058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.641194 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.641195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.641925 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.641926 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642805 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.642905 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.642920 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.642989 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.643040 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":61,"threads_started":1}
I20260925 14:45:10.643466 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34196 (local address 127.21.205.194:40317)
0925 14:45:10.639443 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.639448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.639460 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:10.639502 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.639505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.639544 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.639634 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:10.639793 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.640900 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642107 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642784 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642841 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.642902 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.643331 (+ 429us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.643373 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260925 14:45:10.643484 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.21.205.193:39147)
0925 14:45:10.639498 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.639502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.639511 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:10.639547 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.639549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.639563 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.639639 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:10.639831 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.640645 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642312 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642823 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642907 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.642956 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.643182 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.643391 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":85,"threads_started":1}
I20260925 14:45:10.643671 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:39147 (local address 127.0.0.1:41408)
0925 14:45:10.639362 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.639372 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.639412 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.639657 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.639658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.639659 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:10.639720 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.639722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.640738 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.640740 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.642214 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.642216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642988 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.643043 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.643046 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.643091 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.643123 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":32,"thread_start_us":47,"threads_started":1}
I20260925 14:45:10.643726 26943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.639076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:40317 (local address 127.0.0.1:34196)
0925 14:45:10.639283 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.639320 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.639388 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.639648 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.639651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.639653 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:10.639714 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.639715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.640996 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.640997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.642038 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.642040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.642942 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.642943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.643036 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.643040 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.643091 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.643143 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1,"thread_start_us":51,"threads_started":1}
I20260925 14:45:10.643875 26868 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef238e2484749318c47bfd563b81b44 (DEFAULT_TABLE table=test-table [id=a9f772b440374077a8ec3e41fcd45306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:10.644230 26868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef238e2484749318c47bfd563b81b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:10.644317 26599 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef238e2484749318c47bfd563b81b44 (DEFAULT_TABLE table=test-table [id=a9f772b440374077a8ec3e41fcd45306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:10.644388 26734 tablet_service.cc:1467] Processing CreateTablet for tablet 8ef238e2484749318c47bfd563b81b44 (DEFAULT_TABLE table=test-table [id=a9f772b440374077a8ec3e41fcd45306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:10.644598 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef238e2484749318c47bfd563b81b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:10.644620 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ef238e2484749318c47bfd563b81b44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:10.646795 26948 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Bootstrap starting.
I20260925 14:45:10.647442 26948 tablet_bootstrap.cc:654] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:10.647461 26949 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Bootstrap starting.
I20260925 14:45:10.647964 26950 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Bootstrap starting.
I20260925 14:45:10.648142 26948 log.cc:824] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:10.648429 26949 tablet_bootstrap.cc:654] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:10.648492 26950 tablet_bootstrap.cc:654] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:10.648962 26948 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: No bootstrap required, opened a new log
I20260925 14:45:10.648967 26950 log.cc:824] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:10.648975 26949 log.cc:824] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:10.649030 26948 ts_tablet_manager.cc:1397] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:45:10.649657 26950 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: No bootstrap required, opened a new log
I20260925 14:45:10.649734 26950 ts_tablet_manager.cc:1397] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:10.649888 26949 tablet_bootstrap.cc:492] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: No bootstrap required, opened a new log
I20260925 14:45:10.649955 26949 ts_tablet_manager.cc:1397] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 14:45:10.650843 26948 raft_consensus.cc:348] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.650998 26948 raft_consensus.cc:374] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:10.651041 26948 raft_consensus.cc:729] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b78ba68b46f456eb136b90df07f034e, State: Initialized, Role: FOLLOWER
I20260925 14:45:10.651161 26948 consensus_queue.cc:260] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.651430 26948 ts_tablet_manager.cc:1428] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 14:45:10.651556 26800 heartbeater.cc:503] Master 127.21.205.254:34527 was elected leader, sending a full tablet report...
I20260925 14:45:10.651742 26950 raft_consensus.cc:348] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.651921 26950 raft_consensus.cc:374] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:10.651966 26950 raft_consensus.cc:729] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 326fbfa0d9d5419db4229486996ced25, State: Initialized, Role: FOLLOWER
I20260925 14:45:10.651958 26949 raft_consensus.cc:348] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.652110 26949 raft_consensus.cc:374] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:10.652071 26950 consensus_queue.cc:260] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.652151 26949 raft_consensus.cc:729] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd145f84882455daaa023618dbb9af2, State: Initialized, Role: FOLLOWER
I20260925 14:45:10.652252 26949 consensus_queue.cc:260] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.652328 26950 ts_tablet_manager.cc:1428] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:10.652509 26666 heartbeater.cc:503] Master 127.21.205.254:34527 was elected leader, sending a full tablet report...
I20260925 14:45:10.652575 26949 ts_tablet_manager.cc:1428] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260925 14:45:10.652696 26934 heartbeater.cc:503] Master 127.21.205.254:34527 was elected leader, sending a full tablet report...
W20260925 14:45:10.717011 26801 tablet.cc:2369] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:10.789657 26954 raft_consensus.cc:482] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:10.789753 26954 raft_consensus.cc:504] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.790175 26954 leader_election.cc:288] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 326fbfa0d9d5419db4229486996ced25 (127.21.205.193:39147), 2bd145f84882455daaa023618dbb9af2 (127.21.205.195:35571)
I20260925 14:45:10.794068 26946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.790527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:46419 (local address 127.21.205.195:35571)
0925 14:45:10.790595 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.790597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.790605 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:10.790632 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.790633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.790643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.790704 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:10.790886 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.791634 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793139 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.793720 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793786 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.793794 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.793961 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.794000 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:45:10.794068 26945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.790440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:33271 (local address 127.21.205.193:39147)
0925 14:45:10.790487 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.790489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.790574 (+ 85us) server_negotiation.cc:407] Connection header received
0925 14:45:10.790632 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.790633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.790643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.790704 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:10.790887 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.791621 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793135 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.793704 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793786 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.793794 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.793961 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.793999 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:45:10.794078 26802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.790334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:35571 (local address 127.21.205.194:46419)
0925 14:45:10.790460 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.790474 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.790530 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.790721 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.790722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.790724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:10.790807 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.790809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.791707 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.791708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.793041 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.793044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.793794 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.793880 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.793884 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.793941 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.793988 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":72}
I20260925 14:45:10.794086 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.790334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:39147 (local address 127.21.205.194:33271)
0925 14:45:10.790513 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.790523 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:10.790572 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:10.790721 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:10.790722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:10.790724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:10.790809 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.790810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.791707 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.791708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.793041 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:10.793044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.793794 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.793795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:10.793880 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.793884 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:10.793941 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:45:10.793988 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":95,"thread_start_us":102,"threads_started":1}
I20260925 14:45:10.794441 26888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef238e2484749318c47bfd563b81b44" candidate_uuid: "8b78ba68b46f456eb136b90df07f034e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd145f84882455daaa023618dbb9af2" is_pre_election: true
I20260925 14:45:10.794476 26620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef238e2484749318c47bfd563b81b44" candidate_uuid: "8b78ba68b46f456eb136b90df07f034e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326fbfa0d9d5419db4229486996ced25" is_pre_election: true
I20260925 14:45:10.794560 26888 raft_consensus.cc:2454] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b78ba68b46f456eb136b90df07f034e in term 0.
I20260925 14:45:10.794669 26620 raft_consensus.cc:2454] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b78ba68b46f456eb136b90df07f034e in term 0.
I20260925 14:45:10.794790 26688 leader_election.cc:302] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bd145f84882455daaa023618dbb9af2, 8b78ba68b46f456eb136b90df07f034e; no voters:
I20260925 14:45:10.794906 26954 raft_consensus.cc:2781] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:10.794972 26954 raft_consensus.cc:482] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:10.795002 26954 raft_consensus.cc:3037] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:10.795639 26954 raft_consensus.cc:504] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.795830 26954 leader_election.cc:288] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [CANDIDATE]: Term 1 election: Requested vote from peers 326fbfa0d9d5419db4229486996ced25 (127.21.205.193:39147), 2bd145f84882455daaa023618dbb9af2 (127.21.205.195:35571)
I20260925 14:45:10.795974 26888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef238e2484749318c47bfd563b81b44" candidate_uuid: "8b78ba68b46f456eb136b90df07f034e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bd145f84882455daaa023618dbb9af2"
I20260925 14:45:10.796049 26888 raft_consensus.cc:3037] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:10.796656 26888 raft_consensus.cc:2454] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b78ba68b46f456eb136b90df07f034e in term 1.
I20260925 14:45:10.796811 26688 leader_election.cc:302] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bd145f84882455daaa023618dbb9af2, 8b78ba68b46f456eb136b90df07f034e; no voters:
I20260925 14:45:10.796787 26620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ef238e2484749318c47bfd563b81b44" candidate_uuid: "8b78ba68b46f456eb136b90df07f034e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "326fbfa0d9d5419db4229486996ced25"
I20260925 14:45:10.796864 26620 raft_consensus.cc:3037] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:10.796900 26954 raft_consensus.cc:2781] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:10.797058 26954 raft_consensus.cc:686] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 1 LEADER]: Becoming Leader. State: Replica: 8b78ba68b46f456eb136b90df07f034e, State: Running, Role: LEADER
I20260925 14:45:10.797170 26954 consensus_queue.cc:237] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } }
I20260925 14:45:10.797770 26620 raft_consensus.cc:2454] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b78ba68b46f456eb136b90df07f034e in term 1.
I20260925 14:45:10.798317 26472 catalog_manager.cc:5495] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b78ba68b46f456eb136b90df07f034e (127.21.205.194). New cstate: current_term: 1 leader_uuid: "8b78ba68b46f456eb136b90df07f034e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b78ba68b46f456eb136b90df07f034e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 } health_report { overall_health: UNKNOWN } } }
W20260925 14:45:10.800308 26667 tablet.cc:2369] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:10.825948 26947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.823692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.21.205.194:40317)
0925 14:45:10.823752 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.823754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.823761 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:10.823877 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.823878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.823891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.823947 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:10.824059 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.824493 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.825271 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:10.825503 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:10.825562 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:10.825688 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:10.825769 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:10.825822 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.825877 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.825892 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 14:45:10.827616 26888 raft_consensus.cc:1261] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 1 FOLLOWER]: Refusing update from remote peer 8b78ba68b46f456eb136b90df07f034e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:10.827639 26620 raft_consensus.cc:1261] T 8ef238e2484749318c47bfd563b81b44 P 326fbfa0d9d5419db4229486996ced25 [term 1 FOLLOWER]: Refusing update from remote peer 8b78ba68b46f456eb136b90df07f034e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:10.827999 26958 consensus_queue.cc:1035] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bd145f84882455daaa023618dbb9af2" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:10.828195 26954 consensus_queue.cc:1035] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [LEADER]: Connected to new peer: Peer: permanent_uuid: "326fbfa0d9d5419db4229486996ced25" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 39147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:10.830965 26962 mvcc.cc:205] Tried to move back new op lower bound from 7333263404347305984 to 7333263404225847296. Current Snapshot: MvccSnapshot[applied={T|T < 7333263404347305984}]
I20260925 14:45:10.832260 26473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:10.832350 26473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:10.833076 26473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ca607b6b60a04fcf98880e4076257c4a: Sending DeleteTablet for 3 replicas of tablet 8ef238e2484749318c47bfd563b81b44
I20260925 14:45:10.833401 26734 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef238e2484749318c47bfd563b81b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:10 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:34196
I20260925 14:45:10.833407 26599 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef238e2484749318c47bfd563b81b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:10 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:41408
I20260925 14:45:10.833616 26971 tablet_replica.cc:330] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: stopping tablet replica
I20260925 14:45:10.833606 26868 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ef238e2484749318c47bfd563b81b44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:10 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:48664
I20260925 14:45:10.833711 26971 raft_consensus.cc:2229] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:10.833796 26971 raft_consensus.cc:2258] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:10.833874 26973 tablet_replica.cc:330] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: stopping tablet replica
I20260925 14:45:10.833961 26973 raft_consensus.cc:2229] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:10.834062 26973 raft_consensus.cc:2258] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:10.834203 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26536
I20260925 14:45:10.835187 26971 ts_tablet_manager.cc:1905] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:10.836895 26973 ts_tablet_manager.cc:1905] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:10.837066 26971 ts_tablet_manager.cc:1918] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:10.837141 26971 log.cc:1196] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/8ef238e2484749318c47bfd563b81b44
I20260925 14:45:10.837410 26971 ts_tablet_manager.cc:1939] T 8ef238e2484749318c47bfd563b81b44 P 8b78ba68b46f456eb136b90df07f034e: Deleting consensus metadata
I20260925 14:45:10.837821 26459 catalog_manager.cc:4841] TS 8b78ba68b46f456eb136b90df07f034e (127.21.205.194:40317): tablet 8ef238e2484749318c47bfd563b81b44 (table test-table [id=a9f772b440374077a8ec3e41fcd45306]) successfully deleted
I20260925 14:45:10.838792 26973 ts_tablet_manager.cc:1918] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:10.838877 26973 log.cc:1196] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/8ef238e2484749318c47bfd563b81b44
I20260925 14:45:10.839123 26973 ts_tablet_manager.cc:1939] T 8ef238e2484749318c47bfd563b81b44 P 2bd145f84882455daaa023618dbb9af2: Deleting consensus metadata
I20260925 14:45:10.839488 26458 catalog_manager.cc:4841] TS 2bd145f84882455daaa023618dbb9af2 (127.21.205.195:35571): tablet 8ef238e2484749318c47bfd563b81b44 (table test-table [id=a9f772b440374077a8ec3e41fcd45306]) successfully deleted
W20260925 14:45:10.842796 26460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:39147 (error 108)
I20260925 14:45:10.843097 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26671
I20260925 14:45:10.843302 26942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.843166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:39147 (local address 127.0.0.1:41410)
0925 14:45:10.843225 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:10.843267 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:39147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 14:45:10.848171 26460 catalog_manager.cc:4584] TS 326fbfa0d9d5419db4229486996ced25 (127.21.205.193:39147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8ef238e2484749318c47bfd563b81b44: Network error: Client connection negotiation failed: client connection to 127.21.205.193:39147: connect: Connection refused (error 111)
I20260925 14:45:10.849125 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26805
I20260925 14:45:10.853788 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26442
2026-09-25T14:45:10Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1989 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T14:45:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:10Z Disabled control of system clock
I20260925 14:45:10.880155 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33503
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 14:45:10.956604 26978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:10.956774 26978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:10.956794 26978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:10.957832 26978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:10.957866 26978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:10.957880 26978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:10.957891 26978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:10.957903 26978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:10.959163 26978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:10.959347 26978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:10.959548 26978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:10.964056 26986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.964094 26978 server_base.cc:1034] running on GCE node
W20260925 14:45:10.964104 26984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:10.964037 26983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:10.964556 26978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:10.964869 26978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:10.966003 26978 hybrid_clock.cc:648] HybridClock initialized: now 1790347510965979 us; error 34 us; skew 500 ppm
I20260925 14:45:10.967171 26978 webserver.cc:460] Webserver started at http://127.21.205.254:40743/ using document root <none> and password file <none>
I20260925 14:45:10.967401 26978 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:10.967446 26978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:10.967610 26978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:10.968611 26978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "148459f0881b43e2a25b88ba59cadea2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "c4da18424f9481e9a632a8b4e72777cc"
server_key_iv: "27bfa1fc8c38280d5b3f31a0d547bf52"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.968962 26978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "148459f0881b43e2a25b88ba59cadea2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "c4da18424f9481e9a632a8b4e72777cc"
server_key_iv: "27bfa1fc8c38280d5b3f31a0d547bf52"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.970178 26978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:10.970801 26992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:10.970983 26978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:10.971078 26978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "148459f0881b43e2a25b88ba59cadea2"
format_stamp: "Formatted at 2026-09-25 14:45:10 on dist-test-slave-d7lw"
server_key: "c4da18424f9481e9a632a8b4e72777cc"
server_key_iv: "27bfa1fc8c38280d5b3f31a0d547bf52"
server_key_version: "encryptionkey@0"
I20260925 14:45:10.971161 26978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:10.974309 26978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:10.974555 26978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:10.974689 26978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:10.978338 26978 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33503
I20260925 14:45:10.978365 27046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33503 every 8 connection(s)
I20260925 14:45:10.978655 26978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:10.979254 27047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:10.981736 27047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: Bootstrap starting.
I20260925 14:45:10.982219 27047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:10.982437 27047 log.cc:824] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:10.983016 27047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: No bootstrap required, opened a new log
I20260925 14:45:10.983825 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 26978
I20260925 14:45:10.983916 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:10.984090 22327 external_mini_cluster.cc:1428] Setting key eef0326865beabc38c18829ecd0d5de6
I20260925 14:45:10.985074 27047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } }
I20260925 14:45:10.985230 27047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:10.985270 27047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 148459f0881b43e2a25b88ba59cadea2, State: Initialized, Role: FOLLOWER
I20260925 14:45:10.985414 27047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } }
I20260925 14:45:10.985498 27047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:10.985533 27047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:10.985580 27047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:10.986234 27047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } }
I20260925 14:45:10.986279 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:10.984570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57388 (local address 127.21.205.254:33503)
0925 14:45:10.984757 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:10.984763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:10.984784 (+ 21us) server_negotiation.cc:407] Connection header received
0925 14:45:10.985064 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:10.985069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:10.985101 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:10.985201 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:10.985874 (+ 673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:10.985876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:10.985877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:10.985879 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:10.985902 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:10.985911 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:10.985961 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:10.986049 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 14:45:10.986110 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260925 14:45:10.986315 27047 leader_election.cc:302] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 148459f0881b43e2a25b88ba59cadea2; no voters:
I20260925 14:45:10.986467 27047 leader_election.cc:288] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:10.986704 27047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:10.986742 27052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:10.986883 27052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [term 1 LEADER]: Becoming Leader. State: Replica: 148459f0881b43e2a25b88ba59cadea2, State: Running, Role: LEADER
I20260925 14:45:10.986967 27052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } }
I20260925 14:45:10.987418 27053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "148459f0881b43e2a25b88ba59cadea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } } }
I20260925 14:45:10.987416 27054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 148459f0881b43e2a25b88ba59cadea2. Latest consensus state: current_term: 1 leader_uuid: "148459f0881b43e2a25b88ba59cadea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "148459f0881b43e2a25b88ba59cadea2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33503 } } }
I20260925 14:45:10.987517 27053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:10.987560 27054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:10.987725 27061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:10.988221 27061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:10.989590 27061 catalog_manager.cc:1347] Generated new cluster ID: 161552676abf4ed2a85ff4ce6e9a25f2
I20260925 14:45:10.989643 27061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:11.001217 27061 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:11.001374 27061 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:11.004343 27061 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: Generated new TSK 0
I20260925 14:45:11.004528 27061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:45:11.010326 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--builtin_ntp_servers=127.21.205.212:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 14:45:11.120219 27071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:11.120435 27071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:11.120471 27071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:11.121583 27071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:11.121649 27071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:11.121696 27071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:11.122995 27071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:11.123262 27071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:11.123539 27071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:11.128413 27077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.128559 27071 server_base.cc:1034] running on GCE node
W20260925 14:45:11.128428 27076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.128659 27079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.128904 27071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:11.129174 27071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:11.130326 27071 hybrid_clock.cc:648] HybridClock initialized: now 1790347511130309 us; error 33 us; skew 500 ppm
I20260925 14:45:11.131567 27071 webserver.cc:460] Webserver started at http://127.21.205.193:40879/ using document root <none> and password file <none>
I20260925 14:45:11.131875 27071 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:11.131922 27071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:11.132042 27071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:11.133056 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "a22dd97c408845128997d7b86e353350"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "3494ad09a95933075ccdbcaac5ce4eae"
server_key_iv: "9da37f0774e0f3003852a7509c532c5f"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.133421 27071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "a22dd97c408845128997d7b86e353350"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "3494ad09a95933075ccdbcaac5ce4eae"
server_key_iv: "9da37f0774e0f3003852a7509c532c5f"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.134701 27071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:11.135444 27085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.135637 27071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:11.135730 27071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "a22dd97c408845128997d7b86e353350"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "3494ad09a95933075ccdbcaac5ce4eae"
server_key_iv: "9da37f0774e0f3003852a7509c532c5f"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.135838 27071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:11.138872 27071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:11.139124 27071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:11.139271 27071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:11.139465 27071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:11.139752 27071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:11.139820 27071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.139853 27071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:11.139892 27071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.145045 27071 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37749
I20260925 14:45:11.145153 27200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37749 every 8 connection(s)
I20260925 14:45:11.145407 27071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:45:11.145668 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27071
I20260925 14:45:11.145758 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:11.145946 22327 external_mini_cluster.cc:1428] Setting key 1ebe87238373192d76e79680efe46484
I20260925 14:45:11.146921 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--builtin_ntp_servers=127.21.205.212:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 14:45:11.148870 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.146261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56627 (local address 127.21.205.254:33503)
0925 14:45:11.146302 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.146304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.146506 (+ 202us) server_negotiation.cc:407] Connection header received
0925 14:45:11.147097 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.147098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.147104 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.147132 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.148574 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.148575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.148576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.148581 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.148599 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.148605 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.148697 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.148777 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.148792 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:45:11.149082 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.146175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33503 (local address 127.21.205.193:56627)
0925 14:45:11.146388 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.146425 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.146981 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.147291 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.147295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.147304 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.148408 (+ 1104us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.148409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.148443 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.148476 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.148706 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.148711 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.148714 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.148786 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.148841 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":76,"threads_started":1}
I20260925 14:45:11.149441 27201 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33503
I20260925 14:45:11.149519 27201 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:11.149648 27201 heartbeater.cc:511] Master 127.21.205.254:33503 requested a full tablet report, sending...
I20260925 14:45:11.150066 27009 ts_manager.cc:194] Registered new tserver with Master: a22dd97c408845128997d7b86e353350 (127.21.205.193:37749)
W20260925 14:45:11.225679 27204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:11.225872 27204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:11.225907 27204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:11.227012 27204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:11.227069 27204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:11.227111 27204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:11.228506 27204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:11.228781 27204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:11.229040 27204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:11.233831 27209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.233908 27204 server_base.cc:1034] running on GCE node
W20260925 14:45:11.233843 27210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.233865 27212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.234210 27204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:11.234467 27204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:11.235639 27204 hybrid_clock.cc:648] HybridClock initialized: now 1790347511235627 us; error 41 us; skew 500 ppm
I20260925 14:45:11.236887 27204 webserver.cc:460] Webserver started at http://127.21.205.194:43473/ using document root <none> and password file <none>
I20260925 14:45:11.237124 27204 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:11.237193 27204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:11.237317 27204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:11.238353 27204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "39e0ff23730446539cbb1eb929c9a014"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "6620cfc484bf9f2884f32ab447567894"
server_key_iv: "1ec395c6b0ec30515ce277e7bb184867"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.238689 27204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "39e0ff23730446539cbb1eb929c9a014"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "6620cfc484bf9f2884f32ab447567894"
server_key_iv: "1ec395c6b0ec30515ce277e7bb184867"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.240093 27204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:45:11.240758 27218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.240923 27204 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:11.241014 27204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "39e0ff23730446539cbb1eb929c9a014"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "6620cfc484bf9f2884f32ab447567894"
server_key_iv: "1ec395c6b0ec30515ce277e7bb184867"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.241096 27204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:11.244269 27204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:11.244527 27204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:11.244660 27204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:11.244846 27204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:11.245113 27204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:11.245165 27204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.245201 27204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:11.245227 27204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.250630 27204 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:40485
I20260925 14:45:11.250700 27332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:40485 every 8 connection(s)
I20260925 14:45:11.250978 27204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:11.251773 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27204
I20260925 14:45:11.251886 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:11.252171 22327 external_mini_cluster.cc:1428] Setting key 4c0ae5eeae95b502aed9009e6d7c52be
I20260925 14:45:11.253176 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--builtin_ntp_servers=127.21.205.212:37789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 14:45:11.254135 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.251649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59919 (local address 127.21.205.254:33503)
0925 14:45:11.251717 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.251719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.251950 (+ 231us) server_negotiation.cc:407] Connection header received
0925 14:45:11.252578 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.252579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.252585 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.252645 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.253892 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.253894 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.253894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.253898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.253915 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.253920 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.253981 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.254061 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.254073 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:45:11.254356 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.251612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33503 (local address 127.21.205.194:59919)
0925 14:45:11.251826 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.251869 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.252472 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.252694 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.252697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.252705 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.253727 (+ 1022us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.253728 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.253764 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.253798 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.253993 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.253998 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.254000 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.254072 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.254134 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260925 14:45:11.255076 27333 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33503
I20260925 14:45:11.255170 27333 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:11.255307 27333 heartbeater.cc:511] Master 127.21.205.254:33503 requested a full tablet report, sending...
I20260925 14:45:11.255636 27009 ts_manager.cc:194] Registered new tserver with Master: 39e0ff23730446539cbb1eb929c9a014 (127.21.205.194:40485)
W20260925 14:45:11.330946 27336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:11.331121 27336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:11.331142 27336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:11.332346 27336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:11.332381 27336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:11.332405 27336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:11.333741 27336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33503
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:11.333974 27336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:11.334239 27336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:11.338934 27344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.338918 27341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.338909 27342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.339181 27336 server_base.cc:1034] running on GCE node
I20260925 14:45:11.339411 27336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:11.339660 27336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:11.340793 27336 hybrid_clock.cc:648] HybridClock initialized: now 1790347511340778 us; error 32 us; skew 500 ppm
I20260925 14:45:11.341948 27336 webserver.cc:460] Webserver started at http://127.21.205.195:37917/ using document root <none> and password file <none>
I20260925 14:45:11.342166 27336 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:11.342208 27336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:11.342344 27336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:11.343407 27336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "996dbabb5f2043018f5f12bf9111ca45"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "06b4623674c08eb285e12fbd4dc70f28"
server_key_iv: "2e19d45cdb0f856802c928b730a0c274"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.343742 27336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "996dbabb5f2043018f5f12bf9111ca45"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "06b4623674c08eb285e12fbd4dc70f28"
server_key_iv: "2e19d45cdb0f856802c928b730a0c274"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.345090 27336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:11.345785 27350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.345973 27336 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:11.346094 27336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "996dbabb5f2043018f5f12bf9111ca45"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "06b4623674c08eb285e12fbd4dc70f28"
server_key_iv: "2e19d45cdb0f856802c928b730a0c274"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.346190 27336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:11.349457 27336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:11.349684 27336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:11.349818 27336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:11.350033 27336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:11.350337 27336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:11.350389 27336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.350425 27336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:11.350452 27336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.356451 27336 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:35317
I20260925 14:45:11.356580 27463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:35317 every 8 connection(s)
I20260925 14:45:11.356825 27336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:45:11.358115 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27336
I20260925 14:45:11.358234 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:11.358449 22327 external_mini_cluster.cc:1428] Setting key 2c9e481c5eeaa498afcb059767ed2502
I20260925 14:45:11.359759 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.357523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:60581 (local address 127.21.205.254:33503)
0925 14:45:11.357573 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.357576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.357803 (+ 227us) server_negotiation.cc:407] Connection header received
0925 14:45:11.358181 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.358183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.358191 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.358271 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.359573 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.359573 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.359574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.359578 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.359593 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.359597 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.359651 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.359696 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.359708 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:45:11.359984 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.357508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33503 (local address 127.21.205.195:60581)
0925 14:45:11.357695 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.357723 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.358085 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.358374 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.358378 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.358389 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.359445 (+ 1056us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.359445 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.359471 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.359495 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.359648 (+ 153us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.359651 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.359652 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.359700 (+ 48us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.359740 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":98,"threads_started":1}
I20260925 14:45:11.360358 27464 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33503
I20260925 14:45:11.360447 27464 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:11.360553 27464 heartbeater.cc:511] Master 127.21.205.254:33503 requested a full tablet report, sending...
I20260925 14:45:11.360891 27009 ts_manager.cc:194] Registered new tserver with Master: 996dbabb5f2043018f5f12bf9111ca45 (127.21.205.195:35317)
I20260925 14:45:11.370208 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:45:11.372606 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.371543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57400 (local address 127.21.205.254:33503)
0925 14:45:11.371640 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.371642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.371692 (+ 50us) server_negotiation.cc:407] Connection header received
0925 14:45:11.371758 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.371759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.371764 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.371847 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.372454 (+ 607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.372455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.372456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.372458 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.372469 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.372473 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.372515 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.372548 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.372559 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 14:45:11.374248 27009 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:11.375838 27009 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:11.379719 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.377918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:40485 (local address 127.0.0.1:43322)
0925 14:45:11.378305 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.378315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.378384 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.378666 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.378667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.378670 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.379259 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.379259 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.379264 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.379275 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.379519 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.379525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.379528 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.379612 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.379634 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260925 14:45:11.379770 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.378046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43322 (local address 127.21.205.194:40485)
0925 14:45:11.378244 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.378252 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.378364 (+ 112us) server_negotiation.cc:407] Connection header received
0925 14:45:11.378452 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.378456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.378465 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.378549 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.379339 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.379349 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.379349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.379353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.379384 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.379433 (+ 49us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.379493 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.379645 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.379660 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260925 14:45:11.379937 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.377918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37749 (local address 127.0.0.1:55962)
0925 14:45:11.378084 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.378133 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.378187 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.378632 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.378637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.378643 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.379172 (+ 529us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.379173 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.379185 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.379202 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.379704 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.379705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.379705 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.379772 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.379784 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"spinlock_wait_cycles":768,"thread_start_us":96,"threads_started":1}
I20260925 14:45:11.380136 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.378288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55962 (local address 127.21.205.193:37749)
0925 14:45:11.378475 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.378480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.378490 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:45:11.378517 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.378519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.378528 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.378891 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.379301 (+ 410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.379308 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.379309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.379313 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.379352 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.379379 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.379449 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.380032 (+ 583us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.380049 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":78,"threads_started":1}
I20260925 14:45:11.380198 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.378146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:35317 (local address 127.0.0.1:56344)
0925 14:45:11.378391 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.378400 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.378455 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.379281 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.379282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.379285 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.379702 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.379702 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.379708 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.379719 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.380085 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.380086 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.380086 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.380128 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.380143 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":70,"threads_started":1}
I20260925 14:45:11.380398 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.378233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.21.205.195:35317)
0925 14:45:11.378386 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.378390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.378543 (+ 153us) server_negotiation.cc:407] Connection header received
0925 14:45:11.378570 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.378572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.378580 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.378646 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.379841 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.379847 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.379847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.379851 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.379881 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.379904 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.379976 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.380306 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.380321 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260925 14:45:11.380836 27266 tablet_service.cc:1467] Processing CreateTablet for tablet 05280cf652654f199df4be94a4f01b55 (DEFAULT_TABLE table=test-table [id=5b96042151e0410aa4eb4813cbfbab27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:11.380895 27398 tablet_service.cc:1467] Processing CreateTablet for tablet 05280cf652654f199df4be94a4f01b55 (DEFAULT_TABLE table=test-table [id=5b96042151e0410aa4eb4813cbfbab27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:11.380878 27134 tablet_service.cc:1467] Processing CreateTablet for tablet 05280cf652654f199df4be94a4f01b55 (DEFAULT_TABLE table=test-table [id=5b96042151e0410aa4eb4813cbfbab27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:11.381197 27398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05280cf652654f199df4be94a4f01b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:11.381197 27266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05280cf652654f199df4be94a4f01b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:11.381270 27134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05280cf652654f199df4be94a4f01b55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:11.383822 27479 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Bootstrap starting.
I20260925 14:45:11.384303 27480 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Bootstrap starting.
I20260925 14:45:11.384440 27479 tablet_bootstrap.cc:654] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:11.384577 27481 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Bootstrap starting.
I20260925 14:45:11.384784 27479 log.cc:824] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:11.385097 27480 tablet_bootstrap.cc:654] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:11.385174 27481 tablet_bootstrap.cc:654] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:11.385324 27480 log.cc:824] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:11.385497 27481 log.cc:824] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:11.385699 27479 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: No bootstrap required, opened a new log
I20260925 14:45:11.385811 27479 ts_tablet_manager.cc:1397] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:11.386104 27480 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: No bootstrap required, opened a new log
I20260925 14:45:11.386169 27480 ts_tablet_manager.cc:1397] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:11.387912 27479 raft_consensus.cc:348] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.388053 27479 raft_consensus.cc:374] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:11.388118 27479 raft_consensus.cc:729] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22dd97c408845128997d7b86e353350, State: Initialized, Role: FOLLOWER
I20260925 14:45:11.388131 27481 tablet_bootstrap.cc:492] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: No bootstrap required, opened a new log
I20260925 14:45:11.388186 27481 ts_tablet_manager.cc:1397] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 14:45:11.388264 27479 consensus_queue.cc:260] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.388510 27479 ts_tablet_manager.cc:1428] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:11.388577 27201 heartbeater.cc:503] Master 127.21.205.254:33503 was elected leader, sending a full tablet report...
I20260925 14:45:11.390479 27481 raft_consensus.cc:348] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.390636 27481 raft_consensus.cc:374] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:11.390681 27481 raft_consensus.cc:729] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e0ff23730446539cbb1eb929c9a014, State: Initialized, Role: FOLLOWER
I20260925 14:45:11.390784 27481 consensus_queue.cc:260] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.391047 27481 ts_tablet_manager.cc:1428] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:11.391103 27333 heartbeater.cc:503] Master 127.21.205.254:33503 was elected leader, sending a full tablet report...
I20260925 14:45:11.391757 27480 raft_consensus.cc:348] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.391965 27480 raft_consensus.cc:374] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:11.392021 27480 raft_consensus.cc:729] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996dbabb5f2043018f5f12bf9111ca45, State: Initialized, Role: FOLLOWER
I20260925 14:45:11.392140 27480 consensus_queue.cc:260] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.392391 27480 ts_tablet_manager.cc:1428] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s
I20260925 14:45:11.392455 27464 heartbeater.cc:503] Master 127.21.205.254:33503 was elected leader, sending a full tablet report...
W20260925 14:45:11.396745 27202 tablet.cc:2369] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:11.432569 27487 raft_consensus.cc:482] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:11.432711 27487 raft_consensus.cc:504] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.433056 27487 leader_election.cc:288] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39e0ff23730446539cbb1eb929c9a014 (127.21.205.194:40485), a22dd97c408845128997d7b86e353350 (127.21.205.193:37749)
I20260925 14:45:11.434249 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.433276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:49057 (local address 127.21.205.194:40485)
0925 14:45:11.433333 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.433335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.433344 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:11.433367 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.433368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.433373 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.433432 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.434023 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.434024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.434024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.434028 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.434042 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.434047 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.434118 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.434175 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.434188 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:45:11.434248 27466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.433199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:40485 (local address 127.21.205.195:49057)
0925 14:45:11.433247 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.433259 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.433298 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.433457 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.433459 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.433461 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.433917 (+ 456us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.433918 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.433924 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.433935 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.434116 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.434116 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.434117 (+ 1us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.434172 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.434186 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260925 14:45:11.434451 27478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.433328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:37219 (local address 127.21.205.193:37749)
0925 14:45:11.433373 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.433375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.433493 (+ 118us) server_negotiation.cc:407] Connection header received
0925 14:45:11.433518 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.433519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.433524 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.433580 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.434210 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.434211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.434212 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.434217 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.434233 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.434237 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.434284 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.434353 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.434368 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:11.434449 27488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.433204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37749 (local address 127.21.205.195:37219)
0925 14:45:11.433399 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.433407 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.433451 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.433593 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.433593 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.433595 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:11.434100 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:45:11.434100 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:11.434107 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:45:11.434124 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:11.434291 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:11.434292 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:11.434292 (+ 0us) client_negotiation.cc:769] Sending connection context
0925 14:45:11.434347 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:45:11.434359 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":59,"thread_start_us":93,"threads_started":1}
I20260925 14:45:11.434558 27286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05280cf652654f199df4be94a4f01b55" candidate_uuid: "996dbabb5f2043018f5f12bf9111ca45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e0ff23730446539cbb1eb929c9a014" is_pre_election: true
I20260925 14:45:11.434667 27286 raft_consensus.cc:2454] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 996dbabb5f2043018f5f12bf9111ca45 in term 0.
I20260925 14:45:11.434873 27353 leader_election.cc:302] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39e0ff23730446539cbb1eb929c9a014, 996dbabb5f2043018f5f12bf9111ca45; no voters:
I20260925 14:45:11.434861 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05280cf652654f199df4be94a4f01b55" candidate_uuid: "996dbabb5f2043018f5f12bf9111ca45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22dd97c408845128997d7b86e353350" is_pre_election: true
I20260925 14:45:11.434993 27154 raft_consensus.cc:2454] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 996dbabb5f2043018f5f12bf9111ca45 in term 0.
I20260925 14:45:11.434983 27487 raft_consensus.cc:2781] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:11.435034 27487 raft_consensus.cc:482] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:11.435072 27487 raft_consensus.cc:3037] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:11.435834 27487 raft_consensus.cc:504] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.435961 27487 leader_election.cc:288] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [CANDIDATE]: Term 1 election: Requested vote from peers 39e0ff23730446539cbb1eb929c9a014 (127.21.205.194:40485), a22dd97c408845128997d7b86e353350 (127.21.205.193:37749)
I20260925 14:45:11.436102 27286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05280cf652654f199df4be94a4f01b55" candidate_uuid: "996dbabb5f2043018f5f12bf9111ca45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39e0ff23730446539cbb1eb929c9a014"
I20260925 14:45:11.436141 27154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05280cf652654f199df4be94a4f01b55" candidate_uuid: "996dbabb5f2043018f5f12bf9111ca45" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22dd97c408845128997d7b86e353350"
I20260925 14:45:11.436204 27154 raft_consensus.cc:3037] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:11.436208 27286 raft_consensus.cc:3037] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:11.436825 27154 raft_consensus.cc:2454] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996dbabb5f2043018f5f12bf9111ca45 in term 1.
I20260925 14:45:11.436975 27286 raft_consensus.cc:2454] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996dbabb5f2043018f5f12bf9111ca45 in term 1.
I20260925 14:45:11.436985 27352 leader_election.cc:302] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 996dbabb5f2043018f5f12bf9111ca45, a22dd97c408845128997d7b86e353350; no voters:
I20260925 14:45:11.437078 27487 raft_consensus.cc:2781] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:11.437218 27487 raft_consensus.cc:686] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 1 LEADER]: Becoming Leader. State: Replica: 996dbabb5f2043018f5f12bf9111ca45, State: Running, Role: LEADER
I20260925 14:45:11.437312 27487 consensus_queue.cc:237] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } }
I20260925 14:45:11.438062 27009 catalog_manager.cc:5495] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 reported cstate change: term changed from 0 to 1, leader changed from <none> to 996dbabb5f2043018f5f12bf9111ca45 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "996dbabb5f2043018f5f12bf9111ca45" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "996dbabb5f2043018f5f12bf9111ca45" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 35317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 } health_report { overall_health: UNKNOWN } } }
I20260925 14:45:11.452329 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.451450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56360 (local address 127.21.205.195:35317)
0925 14:45:11.451507 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.451509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.451517 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:11.451578 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.451578 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.451589 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.451637 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.452123 (+ 486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.452124 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.452125 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.452129 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.452141 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.452145 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.452199 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.452253 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.452266 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:45:11.453838 27154 raft_consensus.cc:1261] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 1 FOLLOWER]: Refusing update from remote peer 996dbabb5f2043018f5f12bf9111ca45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:11.453846 27286 raft_consensus.cc:1261] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 1 FOLLOWER]: Refusing update from remote peer 996dbabb5f2043018f5f12bf9111ca45: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:11.454116 27487 consensus_queue.cc:1035] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22dd97c408845128997d7b86e353350" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:11.454236 27489 consensus_queue.cc:1035] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39e0ff23730446539cbb1eb929c9a014" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 40485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:11.456852 27494 mvcc.cc:205] Tried to move back new op lower bound from 7333263406912860160 to 7333263406847926272. Current Snapshot: MvccSnapshot[applied={T|T < 7333263406912860160}]
I20260925 14:45:11.458359 27009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:11.458443 27009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:11.459144 27009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 148459f0881b43e2a25b88ba59cadea2: Sending DeleteTablet for 3 replicas of tablet 05280cf652654f199df4be94a4f01b55
I20260925 14:45:11.459475 27398 tablet_service.cc:1514] Processing DeleteTablet for tablet 05280cf652654f199df4be94a4f01b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:11 UTC) from {username='slave'} at 127.0.0.1:56344
I20260925 14:45:11.459452 27266 tablet_service.cc:1514] Processing DeleteTablet for tablet 05280cf652654f199df4be94a4f01b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:11 UTC) from {username='slave'} at 127.0.0.1:43322
I20260925 14:45:11.459491 27134 tablet_service.cc:1514] Processing DeleteTablet for tablet 05280cf652654f199df4be94a4f01b55 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:11 UTC) from {username='slave'} at 127.0.0.1:55962
I20260925 14:45:11.459717 27503 tablet_replica.cc:330] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350: stopping tablet replica
I20260925 14:45:11.459858 27503 raft_consensus.cc:2229] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:11.459956 27503 raft_consensus.cc:2258] T 05280cf652654f199df4be94a4f01b55 P a22dd97c408845128997d7b86e353350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:11.459955 27502 tablet_replica.cc:330] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: stopping tablet replica
I20260925 14:45:11.460070 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27071
I20260925 14:45:11.460887 27502 raft_consensus.cc:2229] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:11.460994 27502 raft_consensus.cc:2258] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:11.461176 27501 tablet_replica.cc:330] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: stopping tablet replica
I20260925 14:45:11.461256 27501 raft_consensus.cc:2229] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:11.461339 27501 raft_consensus.cc:2258] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:11.461753 27502 ts_tablet_manager.cc:1905] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:11.463682 27502 ts_tablet_manager.cc:1918] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:11.463761 27502 log.cc:1196] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/05280cf652654f199df4be94a4f01b55
I20260925 14:45:11.464068 27502 ts_tablet_manager.cc:1939] T 05280cf652654f199df4be94a4f01b55 P 996dbabb5f2043018f5f12bf9111ca45: Deleting consensus metadata
I20260925 14:45:11.464461 26995 catalog_manager.cc:4841] TS 996dbabb5f2043018f5f12bf9111ca45 (127.21.205.195:35317): tablet 05280cf652654f199df4be94a4f01b55 (table test-table [id=5b96042151e0410aa4eb4813cbfbab27]) successfully deleted
I20260925 14:45:11.464761 27501 ts_tablet_manager.cc:1905] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:11.466297 27501 ts_tablet_manager.cc:1918] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:11.466367 27501 log.cc:1196] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/05280cf652654f199df4be94a4f01b55
I20260925 14:45:11.466631 27501 ts_tablet_manager.cc:1939] T 05280cf652654f199df4be94a4f01b55 P 39e0ff23730446539cbb1eb929c9a014: Deleting consensus metadata
I20260925 14:45:11.467127 26995 catalog_manager.cc:4841] TS 39e0ff23730446539cbb1eb929c9a014 (127.21.205.194:40485): tablet 05280cf652654f199df4be94a4f01b55 (table test-table [id=5b96042151e0410aa4eb4813cbfbab27]) successfully deleted
W20260925 14:45:11.467510 26994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:37749 (error 108)
I20260925 14:45:11.468021 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.467893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37749 (local address 127.0.0.1:55976)
0925 14:45:11.467946 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.467986 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 14:45:11.468178 26994 catalog_manager.cc:4584] TS a22dd97c408845128997d7b86e353350 (127.21.205.193:37749): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05280cf652654f199df4be94a4f01b55: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37749: connect: Connection refused (error 111)
I20260925 14:45:11.470920 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27204
I20260925 14:45:11.476771 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27336
I20260925 14:45:11.481419 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 26978
2026-09-25T14:45:11Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (626 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 14:45:11.500661 22327 mini_oidc.cc:152] Starting JWKS server
I20260925 14:45:11.501652 22327 webserver.cc:460] Webserver started at https://127.0.0.1:38263/ using document root <none> and password file <none>
I20260925 14:45:11.501765 22327 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 14:45:11.501935 22327 webserver.cc:460] Webserver started at http://127.0.0.1:44877/ using document root <none> and password file <none>
2026-09-25T14:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:11Z Disabled control of system clock
I20260925 14:45:11.509225 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37255
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:40117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:37255
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38263/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/testchainca.cert with env {}
W20260925 14:45:11.586323 27510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:11.586503 27510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:11.586535 27510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:11.587586 27510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:11.587634 27510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:11.587647 27510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:11.587658 27510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:11.587670 27510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 14:45:11.587682 27510 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 14:45:11.587693 27510 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38263/jwks.json
I20260925 14:45:11.589176 27510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:37255
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38263/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:11.589351 27510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:11.589510 27510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:11.593978 27515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.594005 27516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.594142 27510 server_base.cc:1034] running on GCE node
W20260925 14:45:11.593988 27518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.594556 27510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:11.594880 27510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:11.596055 27510 hybrid_clock.cc:648] HybridClock initialized: now 1790347511596036 us; error 29 us; skew 500 ppm
I20260925 14:45:11.597164 27510 webserver.cc:460] Webserver started at http://127.21.205.254:44637/ using document root <none> and password file <none>
I20260925 14:45:11.597393 27510 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:11.597457 27510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:11.597601 27510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:11.598510 27510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "23378ab81cf1462581ab8628dc8d3f36"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "a79173b4025a305630f1e7500201f8f5"
server_key_iv: "28c3c42103b7238fdc16fb6ffc26d158"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.598850 27510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "23378ab81cf1462581ab8628dc8d3f36"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "a79173b4025a305630f1e7500201f8f5"
server_key_iv: "28c3c42103b7238fdc16fb6ffc26d158"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.600063 27510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:45:11.600669 27524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.600800 27510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.600890 27510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "23378ab81cf1462581ab8628dc8d3f36"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "a79173b4025a305630f1e7500201f8f5"
server_key_iv: "28c3c42103b7238fdc16fb6ffc26d158"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.600960 27510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:11.618449 27510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:11.618700 27510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:11.618842 27510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:11.622568 27510 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:37255
I20260925 14:45:11.622594 27576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:37255 every 8 connection(s)
I20260925 14:45:11.622922 27510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:11.623106 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27510
I20260925 14:45:11.623224 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:11.623418 22327 external_mini_cluster.cc:1428] Setting key 8dbb599e28701a7c1adbcd7a282bd2df
I20260925 14:45:11.624069 27577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:11.626959 27577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36: Bootstrap starting.
I20260925 14:45:11.627552 27577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:11.627892 27577 log.cc:824] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:11.628636 27577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36: No bootstrap required, opened a new log
I20260925 14:45:11.628986 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.623886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.21.205.254:37255)
0925 14:45:11.624110 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.624119 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.624150 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:45:11.624672 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.624680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.624749 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.624896 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.625057 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.626363 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.627187 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.627587 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.627733 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.628379 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.628383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.628383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.628536 (+ 153us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.628566 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.628575 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.628642 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.628729 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.628800 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260925 14:45:11.629814 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:45:11.630653 27577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } }
I20260925 14:45:11.630791 27577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:11.630834 27577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23378ab81cf1462581ab8628dc8d3f36, State: Initialized, Role: FOLLOWER
I20260925 14:45:11.630942 27577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } }
I20260925 14:45:11.631033 27577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:11.631067 27577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:11.631099 27577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:11.631533 27577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } }
I20260925 14:45:11.631620 27577 leader_election.cc:302] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23378ab81cf1462581ab8628dc8d3f36; no voters:
I20260925 14:45:11.631726 27577 leader_election.cc:288] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:11.631807 27582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:11.631928 27582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [term 1 LEADER]: Becoming Leader. State: Replica: 23378ab81cf1462581ab8628dc8d3f36, State: Running, Role: LEADER
I20260925 14:45:11.631997 27582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } }
I20260925 14:45:11.632030 27577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:11.632263 27584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23378ab81cf1462581ab8628dc8d3f36. Latest consensus state: current_term: 1 leader_uuid: "23378ab81cf1462581ab8628dc8d3f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } } }
I20260925 14:45:11.632251 27583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23378ab81cf1462581ab8628dc8d3f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23378ab81cf1462581ab8628dc8d3f36" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37255 } } }
I20260925 14:45:11.632349 27584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:11.632356 27583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:11.632534 27587 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:11.633008 27587 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:11.634507 27587 catalog_manager.cc:1347] Generated new cluster ID: b97bfd422f9040ac83cece3dfe0cf375
I20260925 14:45:11.634568 27587 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:11.652945 27587 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:11.653609 27587 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:11.665095 27587 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23378ab81cf1462581ab8628dc8d3f36: Generated new TSK 0
I20260925 14:45:11.665289 27587 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 14:45:11.690068 27607 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 14:45:11.690148 27607 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 14:45:11.690392 27580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.675264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56144 (local address 127.21.205.254:37255)
0925 14:45:11.675305 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.675307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.675414 (+ 107us) server_negotiation.cc:407] Connection header received
0925 14:45:11.675475 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.675476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.675488 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.675560 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0925 14:45:11.675687 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.676499 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.677204 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.677551 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.677620 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.677778 (+ 158us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 14:45:11.690174 (+ 12396us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 14:45:11.690333 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56144: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 14:45:11.690441 27580 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56144: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 14:45:11.690718 27606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:11.675181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37255 (local address 127.0.0.1:56144)
0925 14:45:11.675355 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.675363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.675413 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.675575 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.675576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.675577 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 14:45:11.675626 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:11.675627 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.676585 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.676586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:11.677129 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:11.677131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.677627 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.677628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:11.677671 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.677686 (+ 15us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 14:45:11.690322 (+ 12636us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 14:45:11.690397 (+ 75us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.205.254:37255: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.205.254:37255: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38263/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 14:45:11.691149 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27510
2026-09-25T14:45:11Z chronyd exiting
I20260925 14:45:11.727038 22327 test_util.cc:175] -----------------------------------------------
I20260925 14:45:11.727105 22327 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790347486435597-22327-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 14:45:11.727723 22327 mini_oidc.cc:152] Starting JWKS server
I20260925 14:45:11.728404 22327 webserver.cc:460] Webserver started at https://127.0.0.1:36413/ using document root <none> and password file <none>
I20260925 14:45:11.728442 22327 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 14:45:11.728582 22327 webserver.cc:460] Webserver started at http://127.0.0.1:33445/ using document root <none> and password file <none>
2026-09-25T14:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:11Z Disabled control of system clock
I20260925 14:45:11.735426 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37865
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:37371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:37865
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36413/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/testchainca.cert with env {}
W20260925 14:45:11.815388 27614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:11.815618 27614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:11.815656 27614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:11.816875 27614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:11.816938 27614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:11.816960 27614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:11.816982 27614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:11.817021 27614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 14:45:11.817047 27614 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 14:45:11.817087 27614 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36413/jwks.json
I20260925 14:45:11.818591 27614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:37865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36413/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:11.818851 27614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:11.819068 27614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:11.823637 27622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.823606 27619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:11.823599 27620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:11.823864 27614 server_base.cc:1034] running on GCE node
I20260925 14:45:11.824097 27614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:11.824366 27614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:11.825521 27614 hybrid_clock.cc:648] HybridClock initialized: now 1790347511825508 us; error 25 us; skew 500 ppm
I20260925 14:45:11.826663 27614 webserver.cc:460] Webserver started at http://127.21.205.254:45509/ using document root <none> and password file <none>
I20260925 14:45:11.826921 27614 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:11.826987 27614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:11.827149 27614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:11.828191 27614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "cc67c59409774010806f5372d4e6498d"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "4a2732109d1c7183639eee0c27749b38"
server_key_iv: "6bdaba255934b938f21398ea75bbf3cf"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.828552 27614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "cc67c59409774010806f5372d4e6498d"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "4a2732109d1c7183639eee0c27749b38"
server_key_iv: "6bdaba255934b938f21398ea75bbf3cf"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.829795 27614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:11.830484 27628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:11.830693 27614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:11.830803 27614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "cc67c59409774010806f5372d4e6498d"
format_stamp: "Formatted at 2026-09-25 14:45:11 on dist-test-slave-d7lw"
server_key: "4a2732109d1c7183639eee0c27749b38"
server_key_iv: "6bdaba255934b938f21398ea75bbf3cf"
server_key_version: "encryptionkey@0"
I20260925 14:45:11.830878 27614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:11.842813 27614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:11.843108 27614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:11.843252 27614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:11.847036 27614 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:37865
I20260925 14:45:11.847069 27680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:37865 every 8 connection(s)
I20260925 14:45:11.847397 27614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:11.848018 27681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:11.849320 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27614
I20260925 14:45:11.849435 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:11.849615 22327 external_mini_cluster.cc:1428] Setting key 600d183ab7365ba949b4c4260d5eb112
I20260925 14:45:11.850904 27681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d: Bootstrap starting.
I20260925 14:45:11.851486 27681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:11.851753 27681 log.cc:824] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:11.852447 27681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d: No bootstrap required, opened a new log
I20260925 14:45:11.854700 27681 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } }
I20260925 14:45:11.854864 27681 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:11.854898 27681 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc67c59409774010806f5372d4e6498d, State: Initialized, Role: FOLLOWER
I20260925 14:45:11.854934 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.850069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.21.205.254:37865)
0925 14:45:11.850251 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.850263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.850284 (+ 21us) server_negotiation.cc:407] Connection header received
0925 14:45:11.850753 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.850759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.850811 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.850915 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:11.851119 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.852085 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.852884 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.853498 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.853639 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.854231 (+ 592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:11.854236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:11.854236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:11.854448 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:11.854486 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:11.854499 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:11.854587 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:11.854663 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 14:45:11.854740 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260925 14:45:11.855175 27681 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } }
I20260925 14:45:11.855268 27681 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:11.855306 27681 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:11.855346 27681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:11.855729 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:45:11.856002 27681 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } }
I20260925 14:45:11.856118 27681 leader_election.cc:302] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc67c59409774010806f5372d4e6498d; no voters:
I20260925 14:45:11.856278 27681 leader_election.cc:288] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:11.856308 27686 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:11.856407 27686 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [term 1 LEADER]: Becoming Leader. State: Replica: cc67c59409774010806f5372d4e6498d, State: Running, Role: LEADER
I20260925 14:45:11.856477 27686 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } }
I20260925 14:45:11.856504 27681 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:11.857059 27688 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc67c59409774010806f5372d4e6498d. Latest consensus state: current_term: 1 leader_uuid: "cc67c59409774010806f5372d4e6498d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } } }
I20260925 14:45:11.857169 27688 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:11.857045 27687 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc67c59409774010806f5372d4e6498d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc67c59409774010806f5372d4e6498d" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37865 } } }
I20260925 14:45:11.857493 27687 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:11.857625 27694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:11.858109 27694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:11.859510 27694 catalog_manager.cc:1347] Generated new cluster ID: affbfa93914745db909269bc4e3a9956
I20260925 14:45:11.859568 27694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:11.892136 27694 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:11.892583 27694 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:11.895207 27694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc67c59409774010806f5372d4e6498d: Generated new TSK 0
I20260925 14:45:11.895358 27694 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 14:45:11.918226 27711 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 14:45:11.918327 27711 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 14:45:11.918653 27684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:11.904848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.21.205.254:37865)
0925 14:45:11.904902 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:11.904904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:11.904997 (+ 93us) server_negotiation.cc:407] Connection header received
0925 14:45:11.905052 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:11.905053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:11.905064 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:11.905128 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0925 14:45:11.905235 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.906027 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.906755 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.907044 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.907112 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.907253 (+ 141us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 14:45:11.918430 (+ 11177us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36413/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 14:45:11.918591 (+ 161us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36440: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36413/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 14:45:11.918700 27684 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36440: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36413/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 14:45:11.918835 27710 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:11.904775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37865 (local address 127.0.0.1:36440)
0925 14:45:11.904939 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:11.904947 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:11.904992 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:11.905133 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:11.905134 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:11.905135 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 14:45:11.905178 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:11.905179 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.906097 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.906097 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:11.906669 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:11.906670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:11.907120 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:11.907121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:11.907164 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:11.907168 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 14:45:11.918538 (+ 11370us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36413/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 14:45:11.918729 (+ 191us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.205.254:37865: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36413/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":91,"threads_started":1}
I20260925 14:45:11.919498 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27614
2026-09-25T14:45:11Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 14:45:11.959924 22327 mini_oidc.cc:152] Starting JWKS server
I20260925 14:45:11.960585 22327 webserver.cc:460] Webserver started at https://127.0.0.1:34003/ using document root <none> and password file <none>
I20260925 14:45:11.960624 22327 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 14:45:11.960803 22327 webserver.cc:460] Webserver started at http://127.0.0.1:36419/ using document root <none> and password file <none>
2026-09-25T14:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:11Z Disabled control of system clock
I20260925 14:45:11.968586 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:46765
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:44185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:46765
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 14:45:12.045368 27717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:12.045534 27717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:12.045553 27717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:12.046581 27717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:12.046612 27717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:12.046624 27717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:12.046636 27717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:12.046648 27717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 14:45:12.046659 27717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 14:45:12.046670 27717 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34003/jwks.json
I20260925 14:45:12.047928 27717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:46765
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:46765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:12.048110 27717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:12.048314 27717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:12.052666 27725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:12.052662 27723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:12.052700 27722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:12.052933 27717 server_base.cc:1034] running on GCE node
I20260925 14:45:12.053164 27717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:12.053488 27717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:12.054662 27717 hybrid_clock.cc:648] HybridClock initialized: now 1790347512054648 us; error 27 us; skew 500 ppm
I20260925 14:45:12.055708 27717 webserver.cc:460] Webserver started at http://127.21.205.254:46707/ using document root <none> and password file <none>
I20260925 14:45:12.055968 27717 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:12.056030 27717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:12.056172 27717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:12.057137 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "2d93b0287e8648128a518fa59aab3c4f"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "8a1797c12d6434ce52b60f62e518828a"
server_key_iv: "ccd1b0e739f94a1bbc45379835e14554"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.057464 27717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "2d93b0287e8648128a518fa59aab3c4f"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "8a1797c12d6434ce52b60f62e518828a"
server_key_iv: "ccd1b0e739f94a1bbc45379835e14554"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.058656 27717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:12.059329 27731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:12.059500 27717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:12.059605 27717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "2d93b0287e8648128a518fa59aab3c4f"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "8a1797c12d6434ce52b60f62e518828a"
server_key_iv: "ccd1b0e739f94a1bbc45379835e14554"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.059685 27717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:12.074110 27717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:12.074378 27717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:12.074548 27717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:12.078305 27717 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:46765
I20260925 14:45:12.078336 27783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:46765 every 8 connection(s)
I20260925 14:45:12.078644 27717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:12.079356 27784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:12.081836 27784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f: Bootstrap starting.
I20260925 14:45:12.082244 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27717
I20260925 14:45:12.082331 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:12.082399 27784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:12.082551 22327 external_mini_cluster.cc:1428] Setting key a03dbdeb074e1ee4789c2548cf32a8a0
I20260925 14:45:12.082746 27784 log.cc:824] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:12.083607 27784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f: No bootstrap required, opened a new log
I20260925 14:45:12.085899 27784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } }
I20260925 14:45:12.086017 27784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:12.086038 27784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d93b0287e8648128a518fa59aab3c4f, State: Initialized, Role: FOLLOWER
I20260925 14:45:12.086165 27784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } }
I20260925 14:45:12.086231 27784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:12.086253 27784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:12.086323 27784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:12.086978 27784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } }
I20260925 14:45:12.087087 27784 leader_election.cc:302] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d93b0287e8648128a518fa59aab3c4f; no voters:
I20260925 14:45:12.087249 27784 leader_election.cc:288] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:12.087316 27789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:12.087463 27789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [term 1 LEADER]: Becoming Leader. State: Replica: 2d93b0287e8648128a518fa59aab3c4f, State: Running, Role: LEADER
I20260925 14:45:12.087525 27784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:12.087550 27789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } }
I20260925 14:45:12.087953 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d93b0287e8648128a518fa59aab3c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } } }
I20260925 14:45:12.088052 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:12.088302 27797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:12.088892 27797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:12.088975 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:12.082969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.21.205.254:46765)
0925 14:45:12.083161 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:12.083170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:12.083215 (+ 45us) server_negotiation.cc:407] Connection header received
0925 14:45:12.083862 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:12.083871 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:12.083940 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:12.084096 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:12.084246 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.085617 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.086612 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.087254 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.087415 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.088245 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:12.088250 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:12.088250 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:12.088440 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:12.088473 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:12.088486 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:12.088541 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:12.088730 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 14:45:12.088795 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260925 14:45:12.087968 27791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d93b0287e8648128a518fa59aab3c4f. Latest consensus state: current_term: 1 leader_uuid: "2d93b0287e8648128a518fa59aab3c4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d93b0287e8648128a518fa59aab3c4f" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 46765 } } }
I20260925 14:45:12.089764 27791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:12.089833 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:45:12.090817 27797 catalog_manager.cc:1347] Generated new cluster ID: 123e9d324feb4e0984c8875620ebd21f
I20260925 14:45:12.090867 27797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:12.123235 27797 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:12.123739 27797 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:12.133255 27797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d93b0287e8648128a518fa59aab3c4f: Generated new TSK 0
I20260925 14:45:12.133442 27797 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:45:12.145808 27788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:12.142312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.21.205.254:46765)
0925 14:45:12.142360 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:12.142362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:12.142444 (+ 82us) server_negotiation.cc:407] Connection header received
0925 14:45:12.142517 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:12.142517 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:12.142529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:12.142583 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0925 14:45:12.142713 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.143181 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.144341 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.144540 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.144601 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.144811 (+ 210us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 14:45:12.145597 (+ 786us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 14:45:12.145739 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 14:45:12.145866 27788 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45070: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 14:45:12.145905 27813 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:12.142211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:46765 (local address 127.0.0.1:45070)
0925 14:45:12.142372 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:12.142383 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:12.142444 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:12.142597 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:12.142598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:12.142599 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 14:45:12.142650 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:12.142652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.143285 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.143286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:12.144256 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:12.144258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.144627 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.144628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:12.144701 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.144706 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 14:45:12.145699 (+ 993us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 14:45:12.145785 (+ 86us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.205.254:46765: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34003/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260925 14:45:12.146147 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27717
E20260925 14:45:12.158457 27814 webserver.cc:533] Webserver: sslize failed:
E20260925 14:45:12.158510 27814 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-25T14:45:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (225 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T14:45:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:12Z Disabled control of system clock
I20260925 14:45:12.192202 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43329
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:42029
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:43329
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/testchainca.cert with env {}
W20260925 14:45:12.270112 27818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:12.270305 27818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:12.270326 27818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:12.271464 27818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:12.271504 27818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:12.271519 27818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:12.271579 27818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:12.271596 27818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 14:45:12.271620 27818 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 14:45:12.271637 27818 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 14:45:12.273195 27818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:42029
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:43329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:12.273450 27818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:12.273667 27818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:12.278383 27826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:12.278414 27824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:12.278414 27823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:12.278577 27818 server_base.cc:1034] running on GCE node
I20260925 14:45:12.278811 27818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:12.279095 27818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:12.280246 27818 hybrid_clock.cc:648] HybridClock initialized: now 1790347512280236 us; error 28 us; skew 500 ppm
I20260925 14:45:12.281476 27818 webserver.cc:460] Webserver started at http://127.21.205.254:41943/ using document root <none> and password file <none>
I20260925 14:45:12.281738 27818 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:12.281783 27818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:12.281951 27818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:12.282938 27818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "e686f942ab0248d2adf140557b4a9271"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "fb706b5b07826a48577f23b193de83df"
server_key_iv: "309c6de201eab6ab5e4d7ae246ddb287"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.283290 27818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "e686f942ab0248d2adf140557b4a9271"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "fb706b5b07826a48577f23b193de83df"
server_key_iv: "309c6de201eab6ab5e4d7ae246ddb287"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.284605 27818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:12.285271 27832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:12.285468 27818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:12.285583 27818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "e686f942ab0248d2adf140557b4a9271"
format_stamp: "Formatted at 2026-09-25 14:45:12 on dist-test-slave-d7lw"
server_key: "fb706b5b07826a48577f23b193de83df"
server_key_iv: "309c6de201eab6ab5e4d7ae246ddb287"
server_key_version: "encryptionkey@0"
I20260925 14:45:12.285655 27818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:12.298592 27818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:12.298861 27818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:12.299007 27818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:12.302892 27818 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:43329
I20260925 14:45:12.302922 27884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:43329 every 8 connection(s)
I20260925 14:45:12.303237 27818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:12.304211 27885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:12.306218 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27818
I20260925 14:45:12.306319 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:12.306520 22327 external_mini_cluster.cc:1428] Setting key d15a41712da840627d55099bb9f4a9f5
I20260925 14:45:12.306546 27885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271: Bootstrap starting.
I20260925 14:45:12.307327 27885 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:12.307631 27885 log.cc:824] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:12.308245 27885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271: No bootstrap required, opened a new log
I20260925 14:45:12.309976 27885 raft_consensus.cc:348] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } }
I20260925 14:45:12.310107 27885 raft_consensus.cc:374] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:12.310135 27885 raft_consensus.cc:729] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e686f942ab0248d2adf140557b4a9271, State: Initialized, Role: FOLLOWER
I20260925 14:45:12.310278 27885 consensus_queue.cc:260] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } }
I20260925 14:45:12.310364 27885 raft_consensus.cc:388] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:12.310387 27885 raft_consensus.cc:482] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:12.310454 27885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:12.311131 27885 raft_consensus.cc:504] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } }
I20260925 14:45:12.311229 27885 leader_election.cc:302] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e686f942ab0248d2adf140557b4a9271; no voters:
I20260925 14:45:12.311399 27885 leader_election.cc:288] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:12.311455 27890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:12.311595 27890 raft_consensus.cc:686] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [term 1 LEADER]: Becoming Leader. State: Replica: e686f942ab0248d2adf140557b4a9271, State: Running, Role: LEADER
I20260925 14:45:12.311651 27885 sys_catalog.cc:564] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:12.311693 27890 consensus_queue.cc:237] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } }
I20260925 14:45:12.312095 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:12.306958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56646 (local address 127.21.205.254:43329)
0925 14:45:12.307143 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:12.307153 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:12.307186 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:45:12.307635 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:12.307642 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:12.307731 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:12.307925 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:12.308134 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.309064 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.309823 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.310342 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.310493 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.311119 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:12.311122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:12.311123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:45:12.311335 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:12.311397 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:45:12.311413 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:12.311480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:12.311754 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 14:45:12.311853 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":58,"threads_started":1}
I20260925 14:45:12.312089 27891 sys_catalog.cc:455] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e686f942ab0248d2adf140557b4a9271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } } }
I20260925 14:45:12.312187 27891 sys_catalog.cc:458] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:12.312075 27890 sys_catalog.cc:455] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e686f942ab0248d2adf140557b4a9271. Latest consensus state: current_term: 1 leader_uuid: "e686f942ab0248d2adf140557b4a9271" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e686f942ab0248d2adf140557b4a9271" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43329 } } }
I20260925 14:45:12.312361 27890 sys_catalog.cc:458] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:12.312938 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:45:12.313158 27897 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:12.313829 27897 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:12.315233 27897 catalog_manager.cc:1347] Generated new cluster ID: cb9ec8d9f61944f299ce20db3f7bc76f
I20260925 14:45:12.315284 27897 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:12.340915 27897 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:12.341446 27897 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:12.350656 27897 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e686f942ab0248d2adf140557b4a9271: Generated new TSK 0
I20260925 14:45:12.350832 27897 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:45:12.464737 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:12.357251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.21.205.254:43329)
0925 14:45:12.357307 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:12.357309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:12.357399 (+ 90us) server_negotiation.cc:407] Connection header received
0925 14:45:12.357439 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:12.357439 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:12.357451 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:12.357526 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0925 14:45:12.357662 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.358455 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.359185 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.359507 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.359573 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.359704 (+ 131us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 14:45:12.464523 (+104819us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 14:45:12.464679 (+ 156us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56660: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":34}
W20260925 14:45:12.464803 27888 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56660: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260925 14:45:12.464910 27914 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:12.357175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:43329 (local address 127.0.0.1:56660)
0925 14:45:12.357340 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:12.357348 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:12.357396 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:12.357531 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:12.357532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:12.357533 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0925 14:45:12.357601 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:12.357603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.358547 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.358547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:12.359095 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:12.359097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:12.359580 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:12.359581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:12.359619 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:12.359622 (+ 3us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 14:45:12.464655 (+105033us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 14:45:12.464773 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.205.254:43329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":95,"threads_started":1}
I20260925 14:45:12.465219 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27818
2026-09-25T14:45:12Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (297 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (84 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:12 dist-test-slave-d7lw krb5kdc[27926](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:12 dist-test-slave-d7lw krb5kdc[27926](info): set up 2 sockets
Sep 25 14:45:12 dist-test-slave-d7lw krb5kdc[27926](info): commencing operation
krb5kdc: starting...
W20260925 14:45:13.666862 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:13 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:13.776757 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37025
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:37025
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:13.856781 27939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:13.857110 27939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:13.857163 27939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:13.858283 27939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:13.858323 27939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:13.858340 27939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:13.858353 27939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:13.858366 27939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:13.859705 27939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:37025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:13.859977 27939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:13.860217 27939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:13.864794 27945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:13.864872 27944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:13.864966 27939 server_base.cc:1034] running on GCE node
W20260925 14:45:13.864840 27947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:13.865310 27939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:13.865603 27939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:13.866783 27939 hybrid_clock.cc:648] HybridClock initialized: now 1790347513866761 us; error 38 us; skew 500 ppm
Sep 25 14:45:13 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347513, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:13.872601 27939 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:13.873004 27939 webserver.cc:460] Webserver started at http://127.21.205.254:38427/ using document root <none> and password file <none>
I20260925 14:45:13.873236 27939 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:13.873330 27939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:13.873484 27939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:13.874449 27939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "6e61597d11924d4e92c1b7ee61c044f2"
format_stamp: "Formatted at 2026-09-25 14:45:13 on dist-test-slave-d7lw"
server_key: "bf2fbdf0bdbe32b3a697af467c5a9787"
server_key_iv: "60dbadacab0fcca4f02f2e6821848918"
server_key_version: "encryptionkey@0"
I20260925 14:45:13.874816 27939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "6e61597d11924d4e92c1b7ee61c044f2"
format_stamp: "Formatted at 2026-09-25 14:45:13 on dist-test-slave-d7lw"
server_key: "bf2fbdf0bdbe32b3a697af467c5a9787"
server_key_iv: "60dbadacab0fcca4f02f2e6821848918"
server_key_version: "encryptionkey@0"
I20260925 14:45:13.876114 27939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:45:13.876835 27954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:13.877019 27939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:13.877097 27939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "6e61597d11924d4e92c1b7ee61c044f2"
format_stamp: "Formatted at 2026-09-25 14:45:13 on dist-test-slave-d7lw"
server_key: "bf2fbdf0bdbe32b3a697af467c5a9787"
server_key_iv: "60dbadacab0fcca4f02f2e6821848918"
server_key_version: "encryptionkey@0"
I20260925 14:45:13.877154 27939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:13.905272 27939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:13.907306 27939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:13.907410 27939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:13.911250 27939 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:37025
I20260925 14:45:13.911285 28006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:37025 every 8 connection(s)
I20260925 14:45:13.911583 27939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:13.912382 28007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:13.914489 28007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2: Bootstrap starting.
I20260925 14:45:13.915005 28007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:13.915231 28007 log.cc:824] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:13.915853 28007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2: No bootstrap required, opened a new log
I20260925 14:45:13.917392 28007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } }
I20260925 14:45:13.917547 28007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:13.917572 28007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e61597d11924d4e92c1b7ee61c044f2, State: Initialized, Role: FOLLOWER
I20260925 14:45:13.917693 28007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } }
I20260925 14:45:13.917784 28007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:13.917804 28007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:13.917830 28007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:13.918237 28007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } }
I20260925 14:45:13.918311 28007 leader_election.cc:302] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e61597d11924d4e92c1b7ee61c044f2; no voters:
I20260925 14:45:13.918421 28007 leader_election.cc:288] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:13.918489 28010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:13.918715 28007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:13.918829 28010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6e61597d11924d4e92c1b7ee61c044f2, State: Running, Role: LEADER
I20260925 14:45:13.918919 28010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } }
I20260925 14:45:13.919345 28012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e61597d11924d4e92c1b7ee61c044f2. Latest consensus state: current_term: 1 leader_uuid: "6e61597d11924d4e92c1b7ee61c044f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } } }
I20260925 14:45:13.919435 28012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:13.919664 28019 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:13.919734 28011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e61597d11924d4e92c1b7ee61c044f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e61597d11924d4e92c1b7ee61c044f2" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37025 } } }
I20260925 14:45:13.919826 28011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:13.920154 28019 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:13.921232 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 27939
I20260925 14:45:13.921329 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:13.921545 22327 external_mini_cluster.cc:1428] Setting key 950597da979418998cbd856c5670bdad
I20260925 14:45:13.921566 28019 catalog_manager.cc:1347] Generated new cluster ID: 786edf3ba2304fd0939ded666641ab7a
I20260925 14:45:13.921607 28019 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:13 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347513, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:13.932587 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:13.921929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33188 (local address 127.21.205.254:37025)
0925 14:45:13.924010 (+ 2081us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:13.924017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:13.924187 (+ 170us) server_negotiation.cc:407] Connection header received
0925 14:45:13.924323 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:13.924328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:13.924367 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:13.924493 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:13.925121 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:13.926016 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:13.926700 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:13.927124 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:13.927240 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:13.929488 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:13.929492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:13.929492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:13.929507 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:13.931112 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:13.931459 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:13.931462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:13.931463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:13.931554 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:13.931756 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:13.931757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:13.931757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:13.932005 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:13.932180 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:13.932255 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 14:45:13.932387 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2049,"thread_start_us":81,"threads_started":1}
I20260925 14:45:13.938993 28019 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:13.939438 28019 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:13.942662 28019 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e61597d11924d4e92c1b7ee61c044f2: Generated new TSK 0
I20260925 14:45:13.942824 28019 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:13.974131 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:14.075071 28033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:14.075260 28033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:14.075295 28033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:14.076438 28033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:14.076476 28033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:14.076500 28033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:14.077939 28033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:14.078158 28033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:14.078383 28033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:14.083014 28038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:14.083025 28041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:14.083218 28033 server_base.cc:1034] running on GCE node
W20260925 14:45:14.083299 28039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:14.083495 28033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:14.083734 28033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:14.084916 28033 hybrid_clock.cc:648] HybridClock initialized: now 1790347514084906 us; error 27 us; skew 500 ppm
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.087189 28033 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:14.087613 28033 webserver.cc:460] Webserver started at http://127.21.205.193:41967/ using document root <none> and password file <none>
I20260925 14:45:14.087891 28033 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:14.087983 28033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:14.088117 28033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:14.089023 28033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "65827c659d414bbda2573ae992467f99"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "9dc51a4fde419d9dcd9d2c693bc5e814"
server_key_iv: "46e340f0b32a44ea8627a3b978b162a4"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.089322 28033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "65827c659d414bbda2573ae992467f99"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "9dc51a4fde419d9dcd9d2c693bc5e814"
server_key_iv: "46e340f0b32a44ea8627a3b978b162a4"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.090444 28033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:14.091059 28048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.091229 28033 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:14.091322 28033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "65827c659d414bbda2573ae992467f99"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "9dc51a4fde419d9dcd9d2c693bc5e814"
server_key_iv: "46e340f0b32a44ea8627a3b978b162a4"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.091406 28033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:14.105181 28033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:14.106717 28033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:14.106860 28033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:14.107049 28033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:14.107323 28033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:14.107375 28033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.107412 28033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:14.107439 28033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.112965 28033 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:38345
I20260925 14:45:14.113008 28162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:38345 every 8 connection(s)
I20260925 14:45:14.113308 28033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:14.118867 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28033
I20260925 14:45:14.118985 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:14.119166 22327 external_mini_cluster.cc:1428] Setting key b7ef3065f46bb7b7e7b7064311efc23e
I20260925 14:45:14.122371 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.113904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37025 (local address 127.21.205.193:58271)
0925 14:45:14.114064 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:14.114095 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:14.114905 (+ 810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:14.115121 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:14.115124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:14.115336 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:14.115496 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.115500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.116200 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.116201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.116967 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.116969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.117294 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.117295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.117385 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.117811 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:14.117818 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:14.119406 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:14.121060 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.121069 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.121071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.121357 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.121557 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.121558 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.121558 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.121665 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.121931 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:14.121935 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:14.122043 (+ 108us) client_negotiation.cc:769] Sending connection context
0925 14:45:14.122121 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 14:45:14.122239 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260925 14:45:14.122397 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.113905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:58271 (local address 127.21.205.254:37025)
0925 14:45:14.113950 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.113953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.114174 (+ 221us) server_negotiation.cc:407] Connection header received
0925 14:45:14.114983 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.114984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.114997 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.115048 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.115565 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.116120 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.117039 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.117225 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.117274 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.119514 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.119515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.119515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.119524 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.120982 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.121428 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.121429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.121429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.121489 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.121728 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.121729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.121729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.121868 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.121936 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.122170 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.122269 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:45:14.122857 28163 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37025
I20260925 14:45:14.122980 28163 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:14.123159 28163 heartbeater.cc:511] Master 127.21.205.254:37025 requested a full tablet report, sending...
I20260925 14:45:14.123683 27971 ts_manager.cc:194] Registered new tserver with Master: 65827c659d414bbda2573ae992467f99 (127.21.205.193:38345)
I20260925 14:45:14.124327 27971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:58271
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:14.142431 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:14.220582 28168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:14.220789 28168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:14.220826 28168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:14.221935 28168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:14.221997 28168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:14.222040 28168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:14.223441 28168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:14.223695 28168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:14.223922 28168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:14.228725 28173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:14.228725 28176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:14.228708 28174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:14.228906 28168 server_base.cc:1034] running on GCE node
I20260925 14:45:14.229099 28168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:14.229310 28168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:14.230444 28168 hybrid_clock.cc:648] HybridClock initialized: now 1790347514230435 us; error 27 us; skew 500 ppm
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.232805 28168 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:14.233251 28168 webserver.cc:460] Webserver started at http://127.21.205.194:36429/ using document root <none> and password file <none>
I20260925 14:45:14.233495 28168 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:14.233538 28168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:14.233693 28168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:14.234632 28168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "cd697eba7d0e47599caab92d3d9c6e3d"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "ccc27da3b93d2cd46d59f7f0f312f965"
server_key_iv: "123294f2ab39dcbf7e6bc227c2001960"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.234966 28168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "cd697eba7d0e47599caab92d3d9c6e3d"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "ccc27da3b93d2cd46d59f7f0f312f965"
server_key_iv: "123294f2ab39dcbf7e6bc227c2001960"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.236266 28168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:45:14.236903 28184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.237057 28168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:14.237140 28168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "cd697eba7d0e47599caab92d3d9c6e3d"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "ccc27da3b93d2cd46d59f7f0f312f965"
server_key_iv: "123294f2ab39dcbf7e6bc227c2001960"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.237219 28168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:14.261633 28168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:14.263329 28168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:14.263517 28168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:14.263738 28168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:14.264040 28168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:14.264093 28168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.264125 28168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:14.264153 28168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.269855 28168 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:34897
I20260925 14:45:14.269903 28298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:34897 every 8 connection(s)
I20260925 14:45:14.270181 28168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:14.276662 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28168
I20260925 14:45:14.276753 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:14.276957 22327 external_mini_cluster.cc:1428] Setting key e6e85789931706fe4773dddad938d34f
I20260925 14:45:14.279592 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.270853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59299 (local address 127.21.205.254:37025)
0925 14:45:14.270941 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.270944 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.271094 (+ 150us) server_negotiation.cc:407] Connection header received
0925 14:45:14.272077 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.272078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.272091 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.272149 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.272742 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.273267 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.274274 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.274492 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.274566 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.276627 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.276629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.276629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.276635 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.278133 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.278617 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.278617 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.278618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.278666 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.278931 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.278933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.278933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.279058 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.279141 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.279412 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.279494 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:45:14.279819 28301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.270800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37025 (local address 127.21.205.194:59299)
0925 14:45:14.270974 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:14.271015 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:14.271971 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:14.272234 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:14.272237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:14.272465 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:14.272647 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.272651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.273360 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.273361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.274182 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.274184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.274585 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.274586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.274710 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.275120 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:14.275127 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:14.276516 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:14.278232 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.278238 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.278240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.278544 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.278725 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.278726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.278726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.278864 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.279140 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:14.279148 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:14.279282 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:45:14.279390 (+ 108us) client_negotiation.cc:240] Negotiation successful
0925 14:45:14.279538 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":85,"threads_started":1}
I20260925 14:45:14.280365 28299 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37025
I20260925 14:45:14.280452 28299 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:14.280646 28299 heartbeater.cc:511] Master 127.21.205.254:37025 requested a full tablet report, sending...
I20260925 14:45:14.281005 27971 ts_manager.cc:194] Registered new tserver with Master: cd697eba7d0e47599caab92d3d9c6e3d (127.21.205.194:34897)
I20260925 14:45:14.281371 27971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:59299
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:14.300618 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:14.379329 28304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:14.379511 28304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:14.379532 28304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:14.380725 28304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:14.380765 28304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:14.380790 28304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:14.382216 28304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:14.382436 28304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:14.382663 28304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:14.387490 28310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:14.387534 28312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:14.387534 28309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:14.387748 28304 server_base.cc:1034] running on GCE node
I20260925 14:45:14.387970 28304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:14.388207 28304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:14.389367 28304 hybrid_clock.cc:648] HybridClock initialized: now 1790347514389357 us; error 38 us; skew 500 ppm
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.391659 28304 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:14.392143 28304 webserver.cc:460] Webserver started at http://127.21.205.195:43713/ using document root <none> and password file <none>
I20260925 14:45:14.392359 28304 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:14.392426 28304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:14.392555 28304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:14.393555 28304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "7df17ee66b384f028b0426e4550f5e98"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "152dc1acde6a3af8b47dd1de65b730e0"
server_key_iv: "9c5872fe5d807e622120c66f34c2d2b5"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.393905 28304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "7df17ee66b384f028b0426e4550f5e98"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "152dc1acde6a3af8b47dd1de65b730e0"
server_key_iv: "9c5872fe5d807e622120c66f34c2d2b5"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.395150 28304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:14.395869 28319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.396040 28304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:14.396143 28304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "7df17ee66b384f028b0426e4550f5e98"
format_stamp: "Formatted at 2026-09-25 14:45:14 on dist-test-slave-d7lw"
server_key: "152dc1acde6a3af8b47dd1de65b730e0"
server_key_iv: "9c5872fe5d807e622120c66f34c2d2b5"
server_key_version: "encryptionkey@0"
I20260925 14:45:14.396227 28304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:14.412774 28304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:14.414453 28304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:14.414608 28304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:14.414819 28304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:14.415104 28304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:14.415158 28304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.415197 28304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:14.415225 28304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:14.421103 28304 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:41007
I20260925 14:45:14.421131 28433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:41007 every 8 connection(s)
I20260925 14:45:14.421425 28304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:45:14.424732 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28304
I20260925 14:45:14.424840 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:14.425030 22327 external_mini_cluster.cc:1428] Setting key 3f07eb86f44010d29e57fbf44f9d1aca
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:14.430375 28030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.422106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:39339 (local address 127.21.205.254:37025)
0925 14:45:14.422179 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.422181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.422349 (+ 168us) server_negotiation.cc:407] Connection header received
0925 14:45:14.423247 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.423248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.423261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.423311 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.423938 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.424437 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.425650 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.425846 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.425935 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.427903 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.427904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.427905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.427913 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.429074 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.429453 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.429454 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.429455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.429514 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.429761 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.429762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.429762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.429873 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.429941 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.430214 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.430290 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:45:14.430548 28436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.422076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37025 (local address 127.21.205.195:39339)
0925 14:45:14.422240 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:14.422271 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:14.423136 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:14.423377 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:14.423385 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:14.423620 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:14.423850 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.423855 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.424525 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.424526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.425557 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:14.425560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.425955 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.425956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:14.426071 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.426540 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:14.426548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:14.427774 (+ 1226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:14.429158 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.429164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.429166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.429377 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.429580 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:14.429580 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:14.429581 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:14.429688 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:14.429945 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:14.429951 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:14.430077 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 14:45:14.430189 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 14:45:14.430332 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260925 14:45:14.430975 28434 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37025
I20260925 14:45:14.431047 28434 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:14.431221 28434 heartbeater.cc:511] Master 127.21.205.254:37025 requested a full tablet report, sending...
I20260925 14:45:14.431563 27971 ts_manager.cc:194] Registered new tserver with Master: 7df17ee66b384f028b0426e4550f5e98 (127.21.205.195:41007)
I20260925 14:45:14.431962 27971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:39339
I20260925 14:45:14.436498 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/test-admin.keytab.
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.452160 22327 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 14:45:14.453758 28441 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:14.452951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57998 (local address 127.21.205.193:38345)
0925 14:45:14.453132 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.453136 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.453145 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:14.453168 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.453170 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.453182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.453254 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.453663 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57998: BlockingRecv error: recv got EOF from 127.0.0.1:57998 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":87,"threads_started":1}
W20260925 14:45:14.453814 22327 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc)
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.454707 22327 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.455515 22327 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:45:14.462805 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.456012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.21.205.193:38345)
0925 14:45:14.456056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.456058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.456070 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:14.456173 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.456173 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.456184 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.456251 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.456641 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.457339 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.458099 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.458416 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.458502 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.460399 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.460406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.460406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.460412 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.461925 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.462223 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.462226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.462226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.462274 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.462412 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.462412 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.462412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.462514 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.462610 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.462675 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.462729 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 14:45:14.464375 28441 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:14.463946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.21.205.193:38345)
0925 14:45:14.464003 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.464005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.464030 (+ 25us) server_negotiation.cc:407] Connection header received
0925 14:45:14.464078 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.464078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.464087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.464130 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.464334 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58006: BlockingRecv error: recv got EOF from 127.0.0.1:58006 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260925 14:45:14.464448 22327 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc)
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.465327 22327 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.466122 22327 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:45:14.472440 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.466542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.21.205.193:38345)
0925 14:45:14.466592 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.466600 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.466660 (+ 60us) server_negotiation.cc:407] Connection header received
0925 14:45:14.466693 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.466693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.466702 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.466744 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.467049 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.467495 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.468192 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.468369 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.468415 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.470186 (+ 1771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.470187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.470188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.470193 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.471466 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.471738 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.471739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.471739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.471861 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.472014 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.472015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.472015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.472091 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.472156 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.472251 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.472306 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260925 14:45:14.473726 28441 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:14.473333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58034 (local address 127.21.205.193:38345)
0925 14:45:14.473366 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.473367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.473379 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:14.473423 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.473423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.473430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.473474 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.473684 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58034: BlockingRecv error: recv got EOF from 127.0.0.1:58034 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 14:45:14.473809 22327 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790347486435597-22327-0/krb5kdc/krb5cc' of type FILE
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.474653 22327 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:14.475440 22327 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[27926](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347514, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:45:14.481523 28441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:14.475881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58038 (local address 127.21.205.193:38345)
0925 14:45:14.475928 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:14.475930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:14.476008 (+ 78us) server_negotiation.cc:407] Connection header received
0925 14:45:14.476041 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:14.476042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:14.476048 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:14.476097 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:14.476394 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.476813 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.477407 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:14.477572 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:14.477620 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:14.479474 (+ 1854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:14.479476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:14.479476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:14.479483 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:14.480761 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.481035 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.481036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.481036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.481079 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:14.481219 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:14.481219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:14.481219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:14.481289 (+ 70us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:14.481346 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:14.481429 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 14:45:14.481475 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 14:45:14.482067 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28033
I20260925 14:45:14.487421 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28168
I20260925 14:45:14.491734 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28304
I20260925 14:45:14.495682 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 27939
2026-09-25T14:45:14Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1859 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[28454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[28454](info): set up 2 sockets
Sep 25 14:45:14 dist-test-slave-d7lw krb5kdc[28454](info): commencing operation
krb5kdc: starting...
W20260925 14:45:15.518949 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:15 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:15Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab.
Entry for principal oryx/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab.
I20260925 14:45:15.628806 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33533
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab
--principal=oryx/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:15.706569 28467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:15.706746 28467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:15.706768 28467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:15.707921 28467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:15.707957 28467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:15.707971 28467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:15.707983 28467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:15.707995 28467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:15.709394 28467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab
--principal=oryx/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:15.709612 28467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:15.709822 28467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:15.714434 28476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:15.714483 28473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:15.714735 28474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:15.714736 28467 server_base.cc:1034] running on GCE node
I20260925 14:45:15.714969 28467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:15.715266 28467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:15.716441 28467 hybrid_clock.cc:648] HybridClock initialized: now 1790347515716433 us; error 45 us; skew 500 ppm
Sep 25 14:45:15 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:15.718755 28467 init.cc:377] Logged in from keytab as oryx/127.21.205.254@KRBTEST.COM (short username oryx)
I20260925 14:45:15.719138 28467 webserver.cc:460] Webserver started at http://127.21.205.254:41739/ using document root <none> and password file <none>
I20260925 14:45:15.719375 28467 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:15.719440 28467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:15.719589 28467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:15.720613 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "e933784179cb47f8b4bf2872e582e17c"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "ed2e7609fe83aeb94999881373bb21c7"
server_key_iv: "8f36c5461e742f9d13905fa39a4cfef9"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.720966 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "e933784179cb47f8b4bf2872e582e17c"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "ed2e7609fe83aeb94999881373bb21c7"
server_key_iv: "8f36c5461e742f9d13905fa39a4cfef9"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.722285 28467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:15.723068 28483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:15.723251 28467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:15.723344 28467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "e933784179cb47f8b4bf2872e582e17c"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "ed2e7609fe83aeb94999881373bb21c7"
server_key_iv: "8f36c5461e742f9d13905fa39a4cfef9"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.723429 28467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:15.730728 28467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:15.732391 28467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:15.732556 28467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:15.736436 28467 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33533
I20260925 14:45:15.736516 28535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33533 every 8 connection(s)
I20260925 14:45:15.736833 28467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:15.737797 28536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:15.739925 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Bootstrap starting.
I20260925 14:45:15.740468 28536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:15.740712 28536 log.cc:824] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:15.741319 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: No bootstrap required, opened a new log
I20260925 14:45:15.742949 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28467
I20260925 14:45:15.743054 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:15.742933 28536 raft_consensus.cc:348] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } }
I20260925 14:45:15.743110 28536 raft_consensus.cc:374] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:15.743157 28536 raft_consensus.cc:729] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e933784179cb47f8b4bf2872e582e17c, State: Initialized, Role: FOLLOWER
I20260925 14:45:15.743268 22327 external_mini_cluster.cc:1428] Setting key c7045c23d4a9849363b3a23959910bed
I20260925 14:45:15.743341 28536 consensus_queue.cc:260] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } }
I20260925 14:45:15.743448 28536 raft_consensus.cc:388] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:15.743479 28536 raft_consensus.cc:482] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:15.743552 28536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:15.744648 28536 raft_consensus.cc:504] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } }
I20260925 14:45:15.744753 28536 leader_election.cc:302] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e933784179cb47f8b4bf2872e582e17c; no voters:
I20260925 14:45:15.744900 28536 leader_election.cc:288] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:15.745016 28541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:15.745160 28536 sys_catalog.cc:564] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:15.745252 28541 raft_consensus.cc:686] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 LEADER]: Becoming Leader. State: Replica: e933784179cb47f8b4bf2872e582e17c, State: Running, Role: LEADER
I20260925 14:45:15.745396 28541 consensus_queue.cc:237] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } }
I20260925 14:45:15.745792 28541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } }
I20260925 14:45:15.745937 28541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:15.746126 28541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e933784179cb47f8b4bf2872e582e17c. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } }
I20260925 14:45:15.746254 28541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:15.746429 28550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:15.748448 28550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 25 14:45:15 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:15.751528 28550 catalog_manager.cc:1347] Generated new cluster ID: 71b940288cd44aefaaa222fd08534d7a
I20260925 14:45:15.751574 28550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:15.755913 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:15.743833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.21.205.254:33533)
0925 14:45:15.744020 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:15.744030 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:15.744296 (+ 266us) server_negotiation.cc:407] Connection header received
0925 14:45:15.744438 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:15.744446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:15.744498 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:15.744639 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:15.745519 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.747032 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.748212 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.748791 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.748922 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:15.752005 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:15.752008 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:15.752009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:15.752019 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:15.754316 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:15.754678 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:15.754681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:15.754681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:15.754781 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:15.754989 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:15.754990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:15.754990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:15.755234 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:15.755392 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:15.755496 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 14:45:15.755702 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260925 14:45:15.759752 28550 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:15.760444 28550 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:15.770537 28550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Generated new TSK 0
I20260925 14:45:15.770717 28550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
I20260925 14:45:15.800590 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:15.880631 28562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:15.880805 28562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:15.880826 28562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:15.881915 28562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:15.881947 28562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:15.881971 28562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:15.883267 28562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:15.883436 28562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:15.883613 28562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:15.888355 28570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:15.888361 28568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:15.888396 28562 server_base.cc:1034] running on GCE node
W20260925 14:45:15.888367 28567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:15.888736 28562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:15.888968 28562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:15.890156 28562 hybrid_clock.cc:648] HybridClock initialized: now 1790347515890144 us; error 42 us; skew 500 ppm
Sep 25 14:45:15 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:15.892696 28562 init.cc:377] Logged in from keytab as oryx/127.21.205.193@KRBTEST.COM (short username oryx)
I20260925 14:45:15.893107 28562 webserver.cc:460] Webserver started at http://127.21.205.193:44311/ using document root <none> and password file <none>
I20260925 14:45:15.893334 28562 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:15.893402 28562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:15.893554 28562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:15.894543 28562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "e72014da9fc04587aec85ceb25d0645f"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "4bcd53f8ac3390c01e5804216e020672"
server_key_iv: "9afd3478e634b7d2df0518925c5b9fab"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.894907 28562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "e72014da9fc04587aec85ceb25d0645f"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "4bcd53f8ac3390c01e5804216e020672"
server_key_iv: "9afd3478e634b7d2df0518925c5b9fab"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.896267 28562 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:15.897028 28577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:15.897217 28562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:15.897312 28562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "e72014da9fc04587aec85ceb25d0645f"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "4bcd53f8ac3390c01e5804216e020672"
server_key_iv: "9afd3478e634b7d2df0518925c5b9fab"
server_key_version: "encryptionkey@0"
I20260925 14:45:15.897392 28562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:15.910022 28562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:15.911664 28562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:15.911863 28562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:15.912070 28562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:15.912357 28562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:15.912408 28562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:15.912446 28562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:15.912473 28562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:15.918179 28562 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:46105
I20260925 14:45:15.918228 28691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:46105 every 8 connection(s)
I20260925 14:45:15.918498 28562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:15 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:15.924678 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28562
I20260925 14:45:15.924794 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:15.925005 22327 external_mini_cluster.cc:1428] Setting key 61e779d28619baea34722e0b44282c58
I20260925 14:45:15.927475 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:15.919082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:54001 (local address 127.21.205.254:33533)
0925 14:45:15.919127 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:15.919130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:15.919324 (+ 194us) server_negotiation.cc:407] Connection header received
0925 14:45:15.919759 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:15.919760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:15.919769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:15.919890 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:15.920741 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.921240 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.922149 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.922336 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.922386 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:15.924318 (+ 1932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:15.924319 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:15.924319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:15.924325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:15.926083 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:15.926473 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:15.926474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:15.926475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:15.926537 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:15.926792 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:15.926793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:15.926794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:15.926937 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:15.927020 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:15.927349 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 14:45:15.927415 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:45:15.927723 28694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:15.919037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:54001)
0925 14:45:15.919198 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:15.919237 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:15.919679 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:15.919958 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:15.919961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:15.920486 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:15.920663 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:15.920668 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.921316 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.921316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:15.922075 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:15.922077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:15.922414 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:15.922415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:15.922509 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:15.922898 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:15.922905 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:15.924237 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:15.926163 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:15.926170 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:15.926173 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:15.926386 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:15.926617 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:15.926618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:15.926618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:15.926724 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:15.927049 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:15.927056 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:15.927207 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 14:45:15.927316 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 14:45:15.927478 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260925 14:45:15.928288 28692 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:15.928377 28692 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:15.928565 28692 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:15.929041 28500 ts_manager.cc:194] Registered new tserver with Master: e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105)
I20260925 14:45:15.929666 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:54001
WARNING: no policy specified for oryx/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
I20260925 14:45:15.948594 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:16.024874 28697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.025040 28697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.025061 28697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.026250 28697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.026286 28697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.026311 28697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:16.027734 28697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:16.027977 28697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.028172 28697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.032742 28703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.032733 28702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.032732 28705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.032831 28697 server_base.cc:1034] running on GCE node
I20260925 14:45:16.033171 28697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.033425 28697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.034605 28697 hybrid_clock.cc:648] HybridClock initialized: now 1790347516034587 us; error 45 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.037040 28697 init.cc:377] Logged in from keytab as oryx/127.21.205.194@KRBTEST.COM (short username oryx)
I20260925 14:45:16.037434 28697 webserver.cc:460] Webserver started at http://127.21.205.194:38019/ using document root <none> and password file <none>
I20260925 14:45:16.037647 28697 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.037712 28697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.037856 28697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:16.038793 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "ede93d07442948a4aca5b0ef07486820"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ff600ce0787d36c4a058841cd5da1a46"
server_key_iv: "849bc748f3c6a4e6dfecdd62d9c5877c"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.039130 28697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "ede93d07442948a4aca5b0ef07486820"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ff600ce0787d36c4a058841cd5da1a46"
server_key_iv: "849bc748f3c6a4e6dfecdd62d9c5877c"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.040513 28697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:16.041177 28712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.041335 28697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.041429 28697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "ede93d07442948a4aca5b0ef07486820"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ff600ce0787d36c4a058841cd5da1a46"
server_key_iv: "849bc748f3c6a4e6dfecdd62d9c5877c"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.041507 28697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.063583 28697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.065302 28697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.065454 28697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.065656 28697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:16.065932 28697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:16.065991 28697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.066027 28697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:16.066054 28697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.071393 28697 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:34351
I20260925 14:45:16.071466 28826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:34351 every 8 connection(s)
I20260925 14:45:16.071722 28697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:16.072515 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28697
I20260925 14:45:16.072651 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:16.072847 22327 external_mini_cluster.cc:1428] Setting key d54a26ca52571cee8a72ae36fff0306c
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.21.205.195@KRBTEST.COM; defaulting to no policy
I20260925 14:45:16.081717 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.072394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:49747 (local address 127.21.205.254:33533)
0925 14:45:16.072463 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.072465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.072689 (+ 224us) server_negotiation.cc:407] Connection header received
0925 14:45:16.073364 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.073365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.073375 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.073440 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.074718 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.075367 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.076384 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.076626 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.076720 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.078747 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.078749 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.078749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.078757 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.080255 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.080697 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.080699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.080699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.080776 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.081076 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.081077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.081078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.081198 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.081277 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.081540 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.081628 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:45:16.081894 28829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.072380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:49747)
0925 14:45:16.072549 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.072593 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.073268 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.073524 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.073528 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.074371 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.074630 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.074636 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.075454 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.075455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.076283 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.076285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.076728 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.076729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.076859 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.077394 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.077401 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.078653 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.080359 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.080366 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.080368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.080587 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.080856 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.080856 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.080857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.081020 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.081269 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.081275 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.081409 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.081519 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.081666 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260925 14:45:16.082309 28827 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:16.082417 28827 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:16.082607 28827 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:16.083011 28500 ts_manager.cc:194] Registered new tserver with Master: ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.195@KRBTEST.COM" created.
I20260925 14:45:16.083500 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:49747
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
I20260925 14:45:16.096696 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:16.173784 28832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.173988 28832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.174024 28832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.175168 28832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.175232 28832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.175274 28832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:16.176749 28832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:16.176996 28832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.177209 28832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.182029 28837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.182027 28840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.182121 28838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.182320 28832 server_base.cc:1034] running on GCE node
I20260925 14:45:16.182503 28832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.182746 28832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.183836 28832 hybrid_clock.cc:648] HybridClock initialized: now 1790347516183819 us; error 37 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.186126 28832 init.cc:377] Logged in from keytab as oryx/127.21.205.195@KRBTEST.COM (short username oryx)
I20260925 14:45:16.186529 28832 webserver.cc:460] Webserver started at http://127.21.205.195:38927/ using document root <none> and password file <none>
I20260925 14:45:16.186775 28832 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.186818 28832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.186973 28832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:16.187969 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "d6d432e02e2947c9b90b2360cd835983"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "dff9a05418f0f2b4c5a163459401e638"
server_key_iv: "897ba8bbefef1e9be865796e69c1ba5a"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.188344 28832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "d6d432e02e2947c9b90b2360cd835983"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "dff9a05418f0f2b4c5a163459401e638"
server_key_iv: "897ba8bbefef1e9be865796e69c1ba5a"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.189670 28832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:16.190342 28847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.190505 28832 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.190599 28832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "d6d432e02e2947c9b90b2360cd835983"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "dff9a05418f0f2b4c5a163459401e638"
server_key_iv: "897ba8bbefef1e9be865796e69c1ba5a"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.190677 28832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.202831 28832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.204541 28832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.204690 28832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.204892 28832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:16.205157 28832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:16.205209 28832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.205245 28832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:16.205271 28832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.210752 28832 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36337
I20260925 14:45:16.210798 28961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36337 every 8 connection(s)
I20260925 14:45:16.211100 28832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:16.219050 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.211818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:55823 (local address 127.21.205.254:33533)
0925 14:45:16.211871 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.211872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.212007 (+ 135us) server_negotiation.cc:407] Connection header received
0925 14:45:16.212420 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.212421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.212429 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.212478 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.213305 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.213777 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.214667 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.214852 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.214900 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.216805 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.216807 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.216807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.216813 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.217965 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.218303 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.218303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.218304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.218350 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.218571 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.218572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.218572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.218680 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.218727 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.218926 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.218980 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260925 14:45:16.219182 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.211754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:55823)
0925 14:45:16.211910 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.211945 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.212340 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.212520 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.212523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.213028 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.213212 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.213216 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.213848 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.213849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.214593 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.214595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.214917 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.214918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.215065 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.215432 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.215439 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.216707 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.218033 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.218038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.218039 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.218240 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.218416 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.218417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.218417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.218519 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.218730 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.218733 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.218826 (+ 93us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.218896 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.218993 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260925 14:45:16.219661 28962 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:16.219734 28962 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:16.219952 28962 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:16.220372 28500 ts_manager.cc:194] Registered new tserver with Master: d6d432e02e2947c9b90b2360cd835983 (127.21.205.195:36337)
I20260925 14:45:16.220765 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28832
I20260925 14:45:16.220847 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:16.220894 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:55823
I20260925 14:45:16.221020 22327 external_mini_cluster.cc:1428] Setting key f5d38a7e32dad89eef8b496fbe2bcc12
I20260925 14:45:16.221910 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.205.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.253.keytab.
Entry for principal oryx/127.21.205.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.253.keytab.
I20260925 14:45:16.244457 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:34557
--webserver_interface=127.21.205.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.253.keytab
--principal=oryx/127.21.205.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33533,127.21.205.253:34557
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:16.323016 28968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.323181 28968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.323201 28968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.324290 28968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:16.324324 28968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.324338 28968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.324352 28968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:16.324363 28968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:16.325677 28968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33533,127.21.205.253:34557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.253.keytab
--principal=oryx/127.21.205.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:34557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.205.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
W20260925 14:45:16.325765 28968 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.205.254:33533,127.21.205.253:34557'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 14:45:16.325858 28968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.326007 28968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.330488 28973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.330524 28976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.330524 28974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.330802 28968 server_base.cc:1034] running on GCE node
I20260925 14:45:16.331068 28968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.331344 28968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.332494 28968 hybrid_clock.cc:648] HybridClock initialized: now 1790347516332474 us; error 31 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.334709 28968 init.cc:377] Logged in from keytab as oryx/127.21.205.253@KRBTEST.COM (short username oryx)
I20260925 14:45:16.335165 28968 webserver.cc:460] Webserver started at http://127.21.205.253:38659/ using document root <none> and password file <none>
I20260925 14:45:16.335371 28968 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.335433 28968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.335574 28968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:16.336603 28968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/instance:
uuid: "f7da9f120f0c46a2bdceedd36d832bff"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "6fa4e64c5ffd3bf479d763816d50327d"
server_key_iv: "909e8882c994b002860d48803c898ec9"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.336938 28968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal/instance:
uuid: "f7da9f120f0c46a2bdceedd36d832bff"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "6fa4e64c5ffd3bf479d763816d50327d"
server_key_iv: "909e8882c994b002860d48803c898ec9"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.338230 28968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:16.338923 28983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.339098 28968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.339195 28968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "f7da9f120f0c46a2bdceedd36d832bff"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "6fa4e64c5ffd3bf479d763816d50327d"
server_key_iv: "909e8882c994b002860d48803c898ec9"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.339278 28968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.355297 28968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.357045 28968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.357203 28968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.361121 28968 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.253:34557
I20260925 14:45:16.361151 29035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.253:34557 every 8 connection(s)
I20260925 14:45:16.361467 28968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
I20260925 14:45:16.362435 29037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:16.363840 29037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:45:16.368646 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 28968
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.253@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:16.372948 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.364521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43056 (local address 127.21.205.254:33533)
0925 14:45:16.364608 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.364610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.364771 (+ 161us) server_negotiation.cc:407] Connection header received
0925 14:45:16.365249 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.365251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.365264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.365311 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.365887 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.366382 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.367370 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.367564 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.367650 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.370049 (+ 2399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.370051 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.370051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.370060 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.371548 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.371999 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.372000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.372001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.372064 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.372319 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.372320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.372320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.372453 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.372502 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.372811 (+ 309us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.372886 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:45:16.373176 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.364475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43056)
0925 14:45:16.364663 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.364695 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.365136 (+ 441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.365388 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.365391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.365618 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.365795 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.365799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.366473 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.366474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.367267 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.367269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.367661 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.367662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.367827 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.368341 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.368349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.369955 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.371639 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.371644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.371646 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.371927 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.372135 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.372136 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.372137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.372256 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.372516 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.372520 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.372672 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.372756 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.372917 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260925 14:45:16.373811 29037 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.253@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
I20260925 14:45:16.378243 29041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.369168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47308 (local address 127.21.205.253:34557)
0925 14:45:16.369313 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.369317 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.369352 (+ 35us) server_negotiation.cc:407] Connection header received
0925 14:45:16.369371 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.369373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.369388 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.369444 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.370075 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.371050 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.372067 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.372514 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.372585 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.375263 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.375269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.375270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.375277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.377054 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.377454 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.377457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.377457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.377506 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.377703 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.377703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.377704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.377844 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.377980 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.378054 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.378131 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260925 14:45:16.380502 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.374043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47314)
0925 14:45:16.374088 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.374098 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.374132 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.374375 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.374376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.374475 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.374514 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.374515 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.375260 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.375261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.375950 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.375952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.376347 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.376348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.376412 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.376749 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.376753 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.378071 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.379725 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.379725 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.379726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.379957 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.380116 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.380117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.380117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.380158 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.380312 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.380312 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.380341 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.380386 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.380450 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:45:16.380672 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.374103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47314 (local address 127.21.205.253:34557)
0925 14:45:16.374251 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.374253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.374260 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:16.374277 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.374278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.374288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.374358 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.374591 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.375170 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.376052 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.376266 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.376346 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.378154 (+ 1808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.378155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.378155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.378160 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.379663 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.380017 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.380018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.380018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.380071 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.380191 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.380191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.380191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.380267 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.380309 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.380400 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.380491 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260925 14:45:16.381899 29037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Bootstrap starting.
I20260925 14:45:16.382477 29037 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:16.382748 29037 log.cc:824] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:16.383404 29037 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: No bootstrap required, opened a new log
I20260925 14:45:16.384819 29037 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } }
I20260925 14:45:16.384928 29037 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:16.384949 29037 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7da9f120f0c46a2bdceedd36d832bff, State: Initialized, Role: FOLLOWER
I20260925 14:45:16.385097 29037 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } }
I20260925 14:45:16.385329 29044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } } }
I20260925 14:45:16.385381 29037 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:16.385439 29044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: FOLLOWER
W20260925 14:45:16.387059 29055 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:45:16.387113 29055 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:45:16.388670 28968 master_runner.cc:428] Detected that this master f7da9f120f0c46a2bdceedd36d832bff is joining an existing cluster
I20260925 14:45:16.388713 28968 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.205.253:34557
I20260925 14:45:16.397162 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.389844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.21.205.253:34557)
0925 14:45:16.389923 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.389925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.390031 (+ 106us) server_negotiation.cc:407] Connection header received
0925 14:45:16.390124 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.390125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.390132 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.390175 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.390625 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.391375 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.392243 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.392532 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.392630 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.393609 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.393611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.393611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.393617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.395750 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.396194 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.396195 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.396195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.396254 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.396466 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.396468 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.396468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.396590 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.396682 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.396986 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.397074 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 14:45:16.397204 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.389723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47318)
0925 14:45:16.389981 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.389990 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.390037 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.390206 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.390206 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.390450 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.390545 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.390547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.391459 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.391460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.392083 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.392085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.392644 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.392645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.392708 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.393156 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.393162 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.393535 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.395872 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.395873 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.395874 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.396101 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.396331 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.396332 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.396332 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.396397 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.396664 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.396665 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.396711 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.396800 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.396916 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":17,"thread_start_us":75,"threads_started":1}
I20260925 14:45:16.398116 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.389723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43060)
0925 14:45:16.389924 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.389934 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.389994 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.390176 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.390177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.390452 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.390545 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.390547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.392352 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.392353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.393177 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.393180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.393663 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.393664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.393729 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.394244 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.394249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.394652 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.396529 (+ 1877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.396530 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.396531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.396751 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.397013 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.397014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.397014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.397086 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.397814 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.397814 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.397866 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.397941 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.398029 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":3,"thread_start_us":76,"threads_started":1}
I20260925 14:45:16.398149 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.389811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.21.205.254:33533)
0925 14:45:16.389898 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.389900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.389980 (+ 80us) server_negotiation.cc:407] Connection header received
0925 14:45:16.390073 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.390074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.390084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.390170 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.391476 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.392258 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.393263 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.393598 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.393663 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.394772 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.394773 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.394773 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.394779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.396469 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.396821 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.396823 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.396823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.396880 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.397188 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.397190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.397190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.397719 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.397763 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.397975 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.398062 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.254@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
I20260925 14:45:16.407037 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.400443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47322 (local address 127.21.205.253:34557)
0925 14:45:16.400499 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.400501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.400527 (+ 26us) server_negotiation.cc:407] Connection header received
0925 14:45:16.400645 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.400646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.400655 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.400698 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.401081 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.401527 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.402292 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.402507 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.402587 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.404553 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.404554 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.404554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.404561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.405884 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.406281 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.406282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.406282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.406335 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.406541 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.406542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.406542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.406640 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.406706 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.406885 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.406956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:45:16.407060 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.400295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47322)
0925 14:45:16.400419 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.400449 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.400566 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.400713 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.400717 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.400963 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.401004 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.401006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.401624 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.401625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.402196 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.402198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.402591 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.402592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.402684 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.403140 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.403149 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.404468 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.405993 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.406000 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.406002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.406195 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.406411 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.406411 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.406412 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.406473 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.406712 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.406719 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.406766 (+ 47us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.406861 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.406967 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
I20260925 14:45:16.407653 28500 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.205.253:34557: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff"
member_type: NON_VOTER
last_known_addr {
host: "127.21.205.253"
port: 34557
}
attrs {
promote: true
}
}
dest_uuid: "e933784179cb47f8b4bf2872e582e17c"
cas_config_opid_index: -1
I20260925 14:45:16.407974 28500 consensus_queue.cc:237] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } }
I20260925 14:45:16.408418 29065 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f7da9f120f0c46a2bdceedd36d832bff (127.21.205.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.408695 29010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:16.409420 29010 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Refusing update from remote peer e933784179cb47f8b4bf2872e582e17c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 14:45:16.409724 29064 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:16.410378 29065 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.205.253:34557
I20260925 14:45:16.410560 28968 master.cc:537] Master@127.21.205.253:34557 shutting down...
I20260925 14:45:16.410521 29064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.410594 29064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:16.411635 29010 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f7da9f120f0c46a2bdceedd36d832bff (127.21.205.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.413204 29044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader e933784179cb47f8b4bf2872e582e17c. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.413260 29067 mvcc.cc:205] Tried to move back new op lower bound from 7333263424594620416 to 7333263424493780992. Current Snapshot: MvccSnapshot[applied={T|T < 7333263424508940288}]
I20260925 14:45:16.413282 29044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.413357 29069 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.413398 29044 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.413440 29069 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.413465 29044 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.413959 28968 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Raft consensus shutting down.
I20260925 14:45:16.414058 28968 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Raft consensus is shut down!
I20260925 14:45:16.414098 28968 tablet_replica.cc:330] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: stopping tablet replica
I20260925 14:45:16.426731 28968 master.cc:552] Master@127.21.205.253:34557 shutdown complete.
I20260925 14:45:16.426786 28968 master_runner.cc:305] Clearing existing system tablet
I20260925 14:45:16.426913 28968 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.426967 28968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.427546 28968 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.428150 29075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.428262 28968 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.428316 28968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.428337 28968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "f7da9f120f0c46a2bdceedd36d832bff"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "6fa4e64c5ffd3bf479d763816d50327d"
server_key_iv: "909e8882c994b002860d48803c898ec9"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.428864 28968 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:16.430141 28968 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 14:45:16.430195 28968 log.cc:1196] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 14:45:16.430413 28968 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Deleting consensus metadata
I20260925 14:45:16.430595 28968 master_runner.cc:315] Copying system tablet from 127.21.205.254:33533
I20260925 14:45:16.431195 28968 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: tablet copy: Beginning tablet copy session from remote peer at address 127.21.205.254:33533
I20260925 14:45:16.437306 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.431471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43062)
0925 14:45:16.431615 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.431624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.431681 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.431840 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.431841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.431996 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.432054 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.432055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.432983 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.432983 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.433510 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.433512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.433966 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.433967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.434014 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.434464 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.434469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.434756 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.436424 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.436424 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.436425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.436578 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.436781 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.436782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.436782 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.436832 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.437084 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.437085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.437120 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.437169 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.437237 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260925 14:45:16.437351 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.431513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.21.205.254:33533)
0925 14:45:16.431558 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.431560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.431691 (+ 131us) server_negotiation.cc:407] Connection header received
0925 14:45:16.431721 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.431722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.431733 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.431838 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.432124 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.432904 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.433585 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.433892 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.433962 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.434836 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.434837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.434838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.434845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.436351 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.436655 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.436656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.436657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.436715 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.436892 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.436893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.436893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.437015 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.437089 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.437192 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.437262 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:45:16.437639 28520 tablet_copy_service.cc:140] P e933784179cb47f8b4bf2872e582e17c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f7da9f120f0c46a2bdceedd36d832bff ({username='oryx', principal='oryx/127.21.205.253@KRBTEST.COM'} at 127.0.0.1:43062)
I20260925 14:45:16.437726 28520 tablet_copy_service.cc:161] P e933784179cb47f8b4bf2872e582e17c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f7da9f120f0c46a2bdceedd36d832bff at {username='oryx', principal='oryx/127.21.205.253@KRBTEST.COM'} at 127.0.0.1:43062: session id = f7da9f120f0c46a2bdceedd36d832bff-00000000000000000000000000000000
I20260925 14:45:16.438534 28520 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 14:45:16.439029 28968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 14:45:16.439766 28968 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: tablet copy: Starting download of 0 data blocks...
I20260925 14:45:16.439961 28968 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: tablet copy: Starting download of 1 WAL segments...
I20260925 14:45:16.440871 28968 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 14:45:16.441594 28520 tablet_copy_service.cc:342] P e933784179cb47f8b4bf2872e582e17c: Request end of tablet copy session f7da9f120f0c46a2bdceedd36d832bff-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.205.253@KRBTEST.COM'} at 127.0.0.1:43062
I20260925 14:45:16.441656 28520 tablet_copy_service.cc:433] P e933784179cb47f8b4bf2872e582e17c: ending tablet copy session f7da9f120f0c46a2bdceedd36d832bff-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f7da9f120f0c46a2bdceedd36d832bff
I20260925 14:45:16.443709 28968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.444731 29082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.444743 29085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.444962 28968 server_base.cc:1034] running on GCE node
W20260925 14:45:16.445112 29083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.445305 28968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.445443 28968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.446568 28968 hybrid_clock.cc:648] HybridClock initialized: now 1790347516446563 us; error 34 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.447736 28968 init.cc:377] Logged in from keytab as oryx/127.21.205.253@KRBTEST.COM (short username oryx)
I20260925 14:45:16.448011 28968 webserver.cc:460] Webserver started at http://127.21.205.253:38011/ using document root <none> and password file <none>
I20260925 14:45:16.448132 28968 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.448176 28968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.448755 28968 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.449273 29092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.449398 28968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.449440 28968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "f7da9f120f0c46a2bdceedd36d832bff"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "6fa4e64c5ffd3bf479d763816d50327d"
server_key_iv: "909e8882c994b002860d48803c898ec9"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.449489 28968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.459710 28968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.461453 28968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.464169 28968 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.253:34557
I20260925 14:45:16.464397 29145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.253:34557 every 8 connection(s)
I20260925 14:45:16.464514 28968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
I20260925 14:45:16.465302 29146 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:45:16.465775 29146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Bootstrap starting.
I20260925 14:45:16.467953 29146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:45:16.468190 29146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: Bootstrap complete.
I20260925 14:45:16.468374 29146 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } }
I20260925 14:45:16.468457 29146 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f7da9f120f0c46a2bdceedd36d832bff, State: Initialized, Role: LEARNER
I20260925 14:45:16.468493 29146 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } }
I20260925 14:45:16.468689 29146 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:16.468683 29149 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } }
I20260925 14:45:16.468875 29149 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.470240 29160 catalog_manager.cc:1259] Loaded cluster ID: 71b940288cd44aefaaa222fd08534d7a
I20260925 14:45:16.470286 29160 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: loading cluster ID for follower catalog manager: success
I20260925 14:45:16.471215 29160 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: acquiring CA information for follower catalog manager: success
I20260925 14:45:16.471478 29160 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:45:16.486348 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.480115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.21.205.253:34557)
0925 14:45:16.480231 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.480233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.480241 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:16.480263 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.480264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.480280 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.480351 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.480851 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.481478 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.482169 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.482376 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.482452 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.483697 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.483699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.483699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.483705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.485203 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.485630 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.485631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.485631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.485677 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.485875 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.485876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.485876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.485964 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.486050 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.486207 (+ 157us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.486267 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.21.205.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.252.keytab.
Entry for principal oryx/127.21.205.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.252.keytab.
I20260925 14:45:16.510092 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:37035
--webserver_interface=127.21.205.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.252.keytab
--principal=oryx/127.21.205.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:16.591856 29164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.592078 29164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.592111 29164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.593351 29164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:16.593408 29164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.593434 29164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.593463 29164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:16.593493 29164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:16.595018 29164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.252.keytab
--principal=oryx/127.21.205.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:37035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.205.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:16.595264 29164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.595484 29164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.600004 29170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.600036 29169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.600194 29172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.600208 29164 server_base.cc:1034] running on GCE node
I20260925 14:45:16.600458 29164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.600808 29164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.601977 29164 hybrid_clock.cc:648] HybridClock initialized: now 1790347516601977 us; error 40 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.604368 29164 init.cc:377] Logged in from keytab as oryx/127.21.205.252@KRBTEST.COM (short username oryx)
I20260925 14:45:16.604786 29164 webserver.cc:460] Webserver started at http://127.21.205.252:34981/ using document root <none> and password file <none>
I20260925 14:45:16.605008 29164 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.605072 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.605202 29164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:16.606177 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/instance:
uuid: "eff2d92b23c34955b614c82235da890f"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ec6ec31c286ce550cbc5f87a0c43ef34"
server_key_iv: "5d57baa471516e8fed6db1bbaa7064e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.606534 29164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal/instance:
uuid: "eff2d92b23c34955b614c82235da890f"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ec6ec31c286ce550cbc5f87a0c43ef34"
server_key_iv: "5d57baa471516e8fed6db1bbaa7064e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.607887 29164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:45:16.608616 29179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.608769 29164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:16.608870 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "eff2d92b23c34955b614c82235da890f"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ec6ec31c286ce550cbc5f87a0c43ef34"
server_key_iv: "5d57baa471516e8fed6db1bbaa7064e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.608951 29164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.631143 29164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.633574 29164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.633747 29164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.637837 29164 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.252:37035
I20260925 14:45:16.638041 29231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.252:37035 every 8 connection(s)
I20260925 14:45:16.638302 29164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
I20260925 14:45:16.639153 29232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:16.640735 29232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:45:16.644871 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 29164
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.252@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:16.650223 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.641557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43072 (local address 127.21.205.254:33533)
0925 14:45:16.641637 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.641639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.641716 (+ 77us) server_negotiation.cc:407] Connection header received
0925 14:45:16.642146 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.642147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.642156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.642207 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.642770 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.643253 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.644360 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.644609 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.644663 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.647139 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.647140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.647141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.647148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.648656 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.649150 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.649151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.649152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.649200 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.649517 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.649518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.649518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.649618 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.649667 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.649997 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.650096 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 14:45:16.650406 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.641388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43072)
0925 14:45:16.641582 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.641627 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.642047 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.642275 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.642279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.642512 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.642698 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.642702 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.643329 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.643330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.644288 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.644290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.644725 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.644726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.644887 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.645334 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.645342 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.647051 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.648759 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.648764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.648765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.649060 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.649301 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.649302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.649302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.649443 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.649682 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.649687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.649838 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.649978 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.650135 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
I20260925 14:45:16.650856 29232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:45:16.654227 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.645490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.21.205.252:37035)
0925 14:45:16.645670 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.645673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.645687 (+ 14us) server_negotiation.cc:407] Connection header received
0925 14:45:16.645704 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.645705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.645719 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.645779 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.646439 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.647291 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.648087 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.648405 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.648496 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.651316 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.651322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.651322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.651328 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.653016 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.653384 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.653386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.653387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.653435 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.653619 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.653620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.653620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.653776 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.653928 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.654011 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.654112 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":68,"threads_started":1}
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.252@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
I20260925 14:45:16.657930 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.651126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47346)
0925 14:45:16.651173 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.651182 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.651216 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.651371 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.651372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.651512 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.651562 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.651563 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.652458 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.652459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.653219 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.653221 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.653699 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.653700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.653769 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.654146 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.654150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.655369 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.656946 (+ 1577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.656947 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.656948 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.657133 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.657338 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.657338 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.657339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.657392 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.657641 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.657642 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.657681 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.657765 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.657849 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 14:45:16.657953 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.651209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47346 (local address 127.21.205.253:34557)
0925 14:45:16.651262 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.651264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.651272 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:16.651300 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.651301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.651313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.651363 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.651665 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.652374 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.653309 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.653632 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.653695 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.655443 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.655444 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.655445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.655452 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.656850 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.657214 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.657215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.657216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.657274 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.657465 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.657466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.657466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.657568 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.657648 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.657789 (+ 141us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.657864 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:16.658347 29232 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 37035 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.252@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
I20260925 14:45:16.663545 29234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.658589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.0.0.1:36604)
0925 14:45:16.658621 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.658631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.658671 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.658779 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.658779 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.658877 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.658901 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.658902 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.659442 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.659443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.659963 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.659965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.660271 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.660271 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.660305 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.660599 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.660602 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.661562 (+ 960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.662858 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.662859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.662859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.663028 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.663195 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.663195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.663195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.663234 (+ 39us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.663370 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.663371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.663397 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.663438 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.663496 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260925 14:45:16.663604 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.658618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36604 (local address 127.21.205.252:37035)
0925 14:45:16.658653 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.658655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.658663 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:16.658728 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.658729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.658735 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.658776 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.658953 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.659381 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.660033 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.660212 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.660262 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.661639 (+ 1377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.661640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.661640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.661645 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.662785 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.663086 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.663086 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.663087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.663147 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.663266 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.663267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.663267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.663338 (+ 71us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.663377 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.663452 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.663538 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":19}
I20260925 14:45:16.664700 29232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Bootstrap starting.
I20260925 14:45:16.665411 29232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:16.665625 29232 log.cc:824] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:16.666237 29232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: No bootstrap required, opened a new log
I20260925 14:45:16.667512 29232 raft_consensus.cc:348] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 37035 } }
I20260925 14:45:16.667616 29232 raft_consensus.cc:374] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:16.667637 29232 raft_consensus.cc:729] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eff2d92b23c34955b614c82235da890f, State: Initialized, Role: FOLLOWER
I20260925 14:45:16.667717 29232 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 37035 } }
I20260925 14:45:16.667999 29232 sys_catalog.cc:564] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:16.667968 29237 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 37035 } } }
I20260925 14:45:16.668069 29237 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: This master's current role is: FOLLOWER
W20260925 14:45:16.669764 29248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:45:16.669816 29248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:45:16.671880 29164 master_runner.cc:428] Detected that this master eff2d92b23c34955b614c82235da890f is joining an existing cluster
I20260925 14:45:16.671936 29164 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.205.252:37035
I20260925 14:45:16.696224 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.673206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36616 (local address 127.21.205.252:37035)
0925 14:45:16.673261 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.673263 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.673416 (+ 153us) server_negotiation.cc:407] Connection header received
0925 14:45:16.673499 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.673500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.673507 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.673531 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.673919 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674596 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675592 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.675885 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675925 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.693253 (+ 17328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.693254 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.693254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.693260 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.695017 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695535 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.695582 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695739 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.695868 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.695907 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.696065 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.696142 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:45:16.696455 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.672912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.21.205.253:34557)
0925 14:45:16.672951 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.672952 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.673016 (+ 64us) server_negotiation.cc:407] Connection header received
0925 14:45:16.673093 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.673094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.673103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.673154 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.673561 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674281 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675287 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.675566 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675602 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.692643 (+ 17041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.692644 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.692645 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.692651 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.694927 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695519 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695520 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.695577 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695823 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.695971 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.696017 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.696272 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.696357 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:45:16.696506 29256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.673253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.0.0.1:36616)
0925 14:45:16.673361 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.673370 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.673436 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.673578 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.673578 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.673777 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.673841 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.673843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674681 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.674681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.675507 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.675509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.675986 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.676058 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.692355 (+ 16297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.692359 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.693079 (+ 720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.695109 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695109 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695463 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.695640 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695641 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695641 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695685 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.695943 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.695943 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.695974 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.696335 (+ 361us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.696415 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260925 14:45:16.696530 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.672846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.21.205.254:33533)
0925 14:45:16.672901 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.672903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.673408 (+ 505us) server_negotiation.cc:407] Connection header received
0925 14:45:16.673504 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.673505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.673528 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.673558 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.673946 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674648 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675600 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.675917 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675961 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.692493 (+ 16532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.692495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.692495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.692502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.695172 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695631 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.695691 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695935 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.695936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.695936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.696070 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.696115 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.696344 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.696416 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:45:16.696879 29257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.672975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43082)
0925 14:45:16.673351 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.673360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.673432 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.673623 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.673623 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.673807 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.673862 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.673864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674733 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.674734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.675506 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.675507 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.676049 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.676049 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.676108 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.676527 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.676531 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.692392 (+ 15861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.695265 (+ 2873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695266 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695543 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.695771 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695771 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695854 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.696166 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.696166 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.696204 (+ 38us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.696612 (+ 408us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.696739 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260925 14:45:16.697542 29255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.672808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47348)
0925 14:45:16.672970 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.672979 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.673038 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.673206 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.673206 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.673403 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.673477 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.673479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.674366 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.674367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.675204 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.675206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.675666 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.675667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.675724 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.676139 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.676143 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.692556 (+ 16413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.695044 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695046 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695423 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.695659 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.695660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.695660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.695720 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.696068 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.696069 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.696128 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.697365 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.697473 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347515, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.254@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
I20260925 14:45:16.705456 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28562
I20260925 14:45:16.705792 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.698940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.0.0.1:36628)
0925 14:45:16.698998 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.699008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.699046 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.699291 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.699292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.699405 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.699438 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.699439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.700318 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.700319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.701168 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.701170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.701492 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.701493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.701557 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.701882 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.701887 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.703139 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.704718 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.704719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.704720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.704885 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.705108 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.705109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.705109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.705165 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.705417 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.705418 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.705461 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.705554 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.705656 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 14:45:16.706542 29235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.699126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.21.205.252:37035)
0925 14:45:16.699179 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.699181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.699188 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:16.699216 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.699217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.699225 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.699288 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.699501 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.700227 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.701243 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.701423 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.701475 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.703227 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.703228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.703229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.703234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.704620 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.704970 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.704971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.704971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.705025 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.705237 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.705238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.705238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.705361 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.705423 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.706421 (+ 998us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.706486 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:45:16.711439 28500 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.205.252:37035: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "eff2d92b23c34955b614c82235da890f"
member_type: NON_VOTER
last_known_addr {
host: "127.21.205.252"
port: 37035
}
attrs {
promote: true
}
}
dest_uuid: "e933784179cb47f8b4bf2872e582e17c"
cas_config_opid_index: 5
I20260925 14:45:16.711654 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:46105
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:33533
--webserver_port=44311
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:45:16.711706 28500 consensus_queue.cc:237] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:16.712244 29064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eff2d92b23c34955b614c82235da890f (127.21.205.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.713229 29064 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.205.252:37035
I20260925 14:45:16.713306 29065 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.713399 29065 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:16.713716 29206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:16.714639 29206 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 1 FOLLOWER]: Refusing update from remote peer e933784179cb47f8b4bf2872e582e17c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 14:45:16.714800 29164 master.cc:537] Master@127.21.205.252:37035 shutting down...
I20260925 14:45:16.717200 29258 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 14:45:16.717772 28487 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260925 14:45:16.718199 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.713223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.21.205.253:34557)
0925 14:45:16.713573 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.713575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.713709 (+ 134us) server_negotiation.cc:407] Connection header received
0925 14:45:16.713732 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.713733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.713755 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.713808 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:16.713986 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.714735 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.716560 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.717187 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.717239 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.717309 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.718070 (+ 761us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.718114 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327}
I20260925 14:45:16.718667 29164 raft_consensus.cc:2229] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 1 FOLLOWER]: Raft consensus shutting down.
W20260925 14:45:16.718648 28487 consensus_peers.cc:487] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c -> Peer eff2d92b23c34955b614c82235da890f (127.21.205.252:37035): Couldn't send request to peer eff2d92b23c34955b614c82235da890f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 14:45:16.718745 29164 raft_consensus.cc:2258] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:16.718786 29164 tablet_replica.cc:330] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: stopping tablet replica
I20260925 14:45:16.719143 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.713121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.0.0.1:47362)
0925 14:45:16.713602 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.713612 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.713660 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.713858 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.713859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.713860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:16.713896 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.713897 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.714950 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.714951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.716445 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.716448 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.717900 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.717901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.717967 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.717982 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.719020 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.719067 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467}
I20260925 14:45:16.719530 29120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Refusing update from remote peer e933784179cb47f8b4bf2872e582e17c: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 14:45:16.719990 29258 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 14:45:16.720698 29120 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER eff2d92b23c34955b614c82235da890f (127.21.205.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.721652 29149 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.721755 29149 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.721649 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: New leader e933784179cb47f8b4bf2872e582e17c. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: NON_VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: true } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.721954 29258 raft_consensus.cc:1050] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: attempting to promote NON_VOTER f7da9f120f0c46a2bdceedd36d832bff to VOTER
I20260925 14:45:16.721964 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.722160 29258 consensus_queue.cc:237] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:16.722656 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.722541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.0.0.1:36638)
0925 14:45:16.722590 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.722630 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:37035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 14:45:16.722714 29120 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 LEARNER]: Refusing update from remote peer e933784179cb47f8b4bf2872e582e17c: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 14:45:16.722875 29064 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 14:45:16.723136 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.723001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.0.0.1:36652)
0925 14:45:16.723091 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.723114 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:37035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260925 14:45:16.723192 28487 consensus_peers.cc:487] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c -> Peer eff2d92b23c34955b614c82235da890f (127.21.205.252:37035): Couldn't send request to peer eff2d92b23c34955b614c82235da890f. Status: Network error: Client connection negotiation failed: client connection to 127.21.205.252:37035: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 14:45:16.723451 29065 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f7da9f120f0c46a2bdceedd36d832bff (127.21.205.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.723606 29120 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f7da9f120f0c46a2bdceedd36d832bff (127.21.205.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.724361 29065 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.724406 29064 sys_catalog.cc:455] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.724444 29065 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:16.724462 29064 sys_catalog.cc:458] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:16.724678 29262 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e933784179cb47f8b4bf2872e582e17c" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.724772 29262 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:45:16.731959 29164 master.cc:552] Master@127.21.205.252:37035 shutdown complete.
I20260925 14:45:16.732013 29164 master_runner.cc:305] Clearing existing system tablet
I20260925 14:45:16.732160 29164 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.732224 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.732833 29164 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:16.733285 29264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.733440 29164 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.733497 29164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.733526 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "eff2d92b23c34955b614c82235da890f"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ec6ec31c286ce550cbc5f87a0c43ef34"
server_key_iv: "5d57baa471516e8fed6db1bbaa7064e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.734213 29164 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:16.735323 29164 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 14:45:16.735404 29164 log.cc:1196] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 14:45:16.735587 29164 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Deleting consensus metadata
I20260925 14:45:16.735719 29164 master_runner.cc:315] Copying system tablet from 127.21.205.254:33533
I20260925 14:45:16.736414 29164 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: tablet copy: Beginning tablet copy session from remote peer at address 127.21.205.254:33533
I20260925 14:45:16.744139 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.738396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43088 (local address 127.21.205.254:33533)
0925 14:45:16.738437 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.738439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.738554 (+ 115us) server_negotiation.cc:407] Connection header received
0925 14:45:16.738622 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.738623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.738632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.738676 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.738977 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.739464 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.740265 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.740471 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.740559 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.741489 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.741490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.741491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.741497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.742910 (+ 1413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.743328 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.743329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.743329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.743385 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.743598 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.743599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.743600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.743708 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.743785 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.743984 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.744059 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:16.744529 29270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.738365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43088)
0925 14:45:16.738501 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.738510 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.738566 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.738678 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.738679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.738856 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.738894 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.738895 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.739559 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.739560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.740171 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.740173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.740562 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.740563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.740634 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.741089 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.741093 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.741371 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.743028 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.743029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.743029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.743226 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.743481 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.743481 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.743481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.743534 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.743776 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.743777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.743869 (+ 92us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.743954 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.744054 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":103,"threads_started":1}
I20260925 14:45:16.744786 28520 tablet_copy_service.cc:140] P e933784179cb47f8b4bf2872e582e17c: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer eff2d92b23c34955b614c82235da890f ({username='oryx', principal='oryx/127.21.205.252@KRBTEST.COM'} at 127.0.0.1:43088)
I20260925 14:45:16.744832 28520 tablet_copy_service.cc:161] P e933784179cb47f8b4bf2872e582e17c: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer eff2d92b23c34955b614c82235da890f at {username='oryx', principal='oryx/127.21.205.252@KRBTEST.COM'} at 127.0.0.1:43088: session id = eff2d92b23c34955b614c82235da890f-00000000000000000000000000000000
I20260925 14:45:16.745316 28520 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 14:45:16.745808 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 14:45:16.746687 29164 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: tablet copy: Starting download of 0 data blocks...
I20260925 14:45:16.746821 29164 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: tablet copy: Starting download of 1 WAL segments...
I20260925 14:45:16.747475 29164 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 14:45:16.748274 28520 tablet_copy_service.cc:342] P e933784179cb47f8b4bf2872e582e17c: Request end of tablet copy session eff2d92b23c34955b614c82235da890f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.205.252@KRBTEST.COM'} at 127.0.0.1:43088
I20260925 14:45:16.748343 28520 tablet_copy_service.cc:433] P e933784179cb47f8b4bf2872e582e17c: ending tablet copy session eff2d92b23c34955b614c82235da890f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer eff2d92b23c34955b614c82235da890f
I20260925 14:45:16.750475 29164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.751461 29271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.751847 29272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.755935 29164 server_base.cc:1034] running on GCE node
W20260925 14:45:16.756053 29274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.756207 29164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.756361 29164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.757514 29164 hybrid_clock.cc:648] HybridClock initialized: now 1790347516757505 us; error 22 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.758960 29164 init.cc:377] Logged in from keytab as oryx/127.21.205.252@KRBTEST.COM (short username oryx)
I20260925 14:45:16.759243 29164 webserver.cc:460] Webserver started at http://127.21.205.252:46861/ using document root <none> and password file <none>
I20260925 14:45:16.759358 29164 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.759392 29164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.759982 29164 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:16.760510 29281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.760655 29164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.760702 29164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "eff2d92b23c34955b614c82235da890f"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ec6ec31c286ce550cbc5f87a0c43ef34"
server_key_iv: "5d57baa471516e8fed6db1bbaa7064e5"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.760761 29164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.768601 29164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.770349 29164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.773464 29164 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.252:37035
I20260925 14:45:16.773756 29164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
I20260925 14:45:16.774012 29333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.252:37035 every 8 connection(s)
I20260925 14:45:16.774530 29334 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:45:16.774916 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Bootstrap starting.
I20260925 14:45:16.777081 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:45:16.777326 29334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: Bootstrap complete.
I20260925 14:45:16.777530 29334 raft_consensus.cc:348] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:16.777616 29334 raft_consensus.cc:729] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eff2d92b23c34955b614c82235da890f, State: Initialized, Role: LEARNER
I20260925 14:45:16.777657 29334 consensus_queue.cc:260] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:16.777822 29334 sys_catalog.cc:564] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:16.777817 29337 sys_catalog.cc:455] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } } }
I20260925 14:45:16.778057 29337 sys_catalog.cc:458] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f [sys.catalog]: This master's current role is: LEARNER
I20260925 14:45:16.779398 29348 catalog_manager.cc:1259] Loaded cluster ID: 71b940288cd44aefaaa222fd08534d7a
I20260925 14:45:16.779485 29348 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: loading cluster ID for follower catalog manager: success
I20260925 14:45:16.780577 29348 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: acquiring CA information for follower catalog manager: success
I20260925 14:45:16.780879 29348 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eff2d92b23c34955b614c82235da890f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 14:45:16.830231 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.830500 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.830530 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.832156 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.832201 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.832234 29259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:16.834340 29259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:46105
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=44311
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:16.834570 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.834824 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.840042 29353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.840090 29354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.840056 29356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.840332 29259 server_base.cc:1034] running on GCE node
I20260925 14:45:16.840546 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.840782 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.841997 29259 hybrid_clock.cc:648] HybridClock initialized: now 1790347516841979 us; error 29 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.844982 29259 init.cc:377] Logged in from keytab as oryx/127.21.205.193@KRBTEST.COM (short username oryx)
I20260925 14:45:16.845485 29259 webserver.cc:460] Webserver started at http://127.21.205.193:44311/ using document root <none> and password file <none>
I20260925 14:45:16.845743 29259 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.845810 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.847476 29259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:45:16.848114 29363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.848299 29259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:45:16.848395 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "e72014da9fc04587aec85ceb25d0645f"
format_stamp: "Formatted at 2026-09-25 14:45:15 on dist-test-slave-d7lw"
server_key: "4bcd53f8ac3390c01e5804216e020672"
server_key_iv: "9afd3478e634b7d2df0518925c5b9fab"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.848773 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.870900 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.872783 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.872889 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.873050 29259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:16.873389 29259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:16.873422 29259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.873445 29259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:16.873489 29259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.879209 29259 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:46105
I20260925 14:45:16.879318 29477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:46105 every 8 connection(s)
I20260925 14:45:16.879621 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:45:16.880057 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 29259
I20260925 14:45:16.880239 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28697
I20260925 14:45:16.887262 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:34351
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:33533
--webserver_port=38019
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
I20260925 14:45:16.891709 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.882327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:50259 (local address 127.21.205.253:34557)
0925 14:45:16.882372 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.882374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.882981 (+ 607us) server_negotiation.cc:407] Connection header received
0925 14:45:16.883033 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.883034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.883046 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.883098 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.883507 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.884265 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885224 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.885496 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885542 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.888372 (+ 2830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.888373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.888373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.888381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.890118 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.890596 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.890597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.890598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.890651 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.890964 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.890965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.890965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.891121 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.891164 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.891539 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.891622 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:16.892144 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.880342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:42975 (local address 127.21.205.254:33533)
0925 14:45:16.880392 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.880395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.881045 (+ 650us) server_negotiation.cc:407] Connection header received
0925 14:45:16.881553 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.881555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.881568 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.881622 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.882832 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.883586 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885085 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.885367 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885405 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.889258 (+ 3853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.889259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.889260 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.889266 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.890967 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.891303 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.891304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.891304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.891356 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.891583 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.891584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.891585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.891707 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.891749 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.891978 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.892066 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:16.892607 29485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.882127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.21.205.193:50259)
0925 14:45:16.882895 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.882906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.882975 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.883141 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.883142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.883357 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.883411 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.883413 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.884360 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.884361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.885149 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.885151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.885610 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.885662 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.886367 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.886377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.888233 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.890234 (+ 2001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.890240 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.890242 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.890512 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.890740 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.890741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.890741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.890889 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.891211 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.891218 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.891377 (+ 159us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.892030 (+ 653us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.892391 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"thread_start_us":93,"threads_started":1}
I20260925 14:45:16.892719 29483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.880354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:42975)
0925 14:45:16.880904 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.880944 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.881452 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.881750 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.881754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.882544 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.882733 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.882737 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.883713 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.883714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.884980 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.884982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.885485 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.885620 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.886367 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.886377 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.889188 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.891041 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.891042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.891042 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.891229 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.891432 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.891433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.891433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.891484 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.891826 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.891827 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.891870 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.892298 (+ 428us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.892571 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":174,"thread_start_us":76,"threads_started":1}
I20260925 14:45:16.893158 29482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.880596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.21.205.193:51937)
0925 14:45:16.881624 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:16.881634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:16.881674 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:16.883232 (+ 1558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:16.883233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:16.883430 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:16.883518 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.883519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.884885 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.884886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.885803 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:16.885805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.886328 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.886329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:16.886403 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.886778 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:16.886783 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:16.889410 (+ 2627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:16.891970 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.891971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.891972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.892224 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.892430 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:16.892431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:16.892431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:16.892489 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:16.892797 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:16.892798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:16.892840 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 14:45:16.892905 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:45:16.893002 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"thread_start_us":92,"threads_started":1}
I20260925 14:45:16.893489 29479 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:16.893586 29479 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:16.893723 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:16.880825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:51937 (local address 127.21.205.252:37035)
0925 14:45:16.883073 (+ 2248us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:16.883075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:16.883082 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:16.883107 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:16.883108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:16.883120 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:16.883172 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:16.883611 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.884778 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.885876 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:16.886216 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:16.886265 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:16.889719 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:16.889720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:16.889721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:16.889728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:16.891877 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.892307 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.892309 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.892309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.892364 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:16.892554 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:16.892555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:16.892555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:16.892709 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:16.892771 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:16.893389 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 14:45:16.893574 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2227,"thread_start_us":56,"threads_started":1}
I20260925 14:45:16.893766 29479 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:16.894156 29480 heartbeater.cc:347] Connected to a master server at 127.21.205.253:34557
I20260925 14:45:16.894196 29478 heartbeater.cc:347] Connected to a master server at 127.21.205.252:37035
I20260925 14:45:16.894209 29480 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:16.894232 28500 ts_manager.cc:194] Re-registered known tserver with Master: e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105)
I20260925 14:45:16.894261 29478 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:16.894306 29480 heartbeater.cc:511] Master 127.21.205.253:34557 requested a full tablet report, sending...
I20260925 14:45:16.894323 29478 heartbeater.cc:511] Master 127.21.205.252:37035 requested a full tablet report, sending...
I20260925 14:45:16.894694 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:42975
I20260925 14:45:16.894735 29109 ts_manager.cc:194] Registered new tserver with Master: e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105)
I20260925 14:45:16.895044 29298 ts_manager.cc:194] Registered new tserver with Master: e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105)
W20260925 14:45:16.968755 29486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:16.968981 29486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:16.969002 29486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:16.970266 29486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:16.970305 29486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:16.970330 29486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:16.971707 29486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:34351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=38019
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:16.971932 29486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:16.972160 29486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:16.976794 29494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.976795 29492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:16.976817 29491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:16.977486 29486 server_base.cc:1034] running on GCE node
I20260925 14:45:16.977675 29486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:16.977941 29486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:16.979108 29486 hybrid_clock.cc:648] HybridClock initialized: now 1790347516979093 us; error 22 us; skew 500 ppm
Sep 25 14:45:16 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:16.981596 29486 init.cc:377] Logged in from keytab as oryx/127.21.205.194@KRBTEST.COM (short username oryx)
I20260925 14:45:16.982120 29486 webserver.cc:460] Webserver started at http://127.21.205.194:38019/ using document root <none> and password file <none>
I20260925 14:45:16.982347 29486 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:16.982403 29486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:16.983667 29486 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:16.984328 29501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.984505 29486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:45:16.984597 29486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "ede93d07442948a4aca5b0ef07486820"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "ff600ce0787d36c4a058841cd5da1a46"
server_key_iv: "849bc748f3c6a4e6dfecdd62d9c5877c"
server_key_version: "encryptionkey@0"
I20260925 14:45:16.984885 29486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:16.993696 29486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:16.995467 29486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:16.995577 29486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:16.995723 29486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:16.996032 29486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:16.996066 29486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:16.996086 29486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:16.996098 29486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:17.001370 29486 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:34351
I20260925 14:45:17.001446 29615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:34351 every 8 connection(s)
I20260925 14:45:17.001732 29486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:17.002691 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 29486
I20260925 14:45:17.002847 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28832
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
I20260925 14:45:17.010018 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:36337
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:33533
--webserver_port=38927
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:33139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347516, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:17.014225 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:37935 (local address 127.21.205.253:34557)
0925 14:45:17.002447 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.002450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.002991 (+ 541us) server_negotiation.cc:407] Connection header received
0925 14:45:17.003595 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.003597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.003609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.003678 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.004634 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005430 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007034 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007360 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007399 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.010607 (+ 3208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.010609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.010609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.010616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.012399 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.012901 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.012902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.012903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.012961 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.013268 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.013270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.013270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.013421 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.013489 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.014006 (+ 517us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.014085 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:45:17.014396 29621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.21.205.194:37935)
0925 14:45:17.002926 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.002947 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.003492 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.003820 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.003824 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.004262 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.004530 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.004535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005563 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.005564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.006938 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.006941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007477 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.007619 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.008356 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.008361 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.010488 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.012478 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.012484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.012486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.012815 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.013028 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.013028 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.013029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.013162 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.013528 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.013534 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.013691 (+ 157us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.013801 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.013995 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"mutex_wait_us":129,"thread_start_us":114,"threads_started":1}
I20260925 14:45:17.014885 29618 heartbeater.cc:347] Connected to a master server at 127.21.205.253:34557
I20260925 14:45:17.015015 29618 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.015204 29618 heartbeater.cc:511] Master 127.21.205.253:34557 requested a full tablet report, sending...
I20260925 14:45:17.014901 29620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:42507)
0925 14:45:17.002774 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.002814 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.003608 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.003877 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.003879 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.004268 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.004531 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.004537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005575 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.005576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.006939 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.006942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007478 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.007620 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.008480 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.008485 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.011616 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.013625 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.013626 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.013626 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.013856 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.014103 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.014104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.014104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.014162 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.014475 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.014476 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.014528 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.014605 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.014734 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":152,"thread_start_us":115,"threads_started":1}
I20260925 14:45:17.015707 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:42507 (local address 127.21.205.254:33533)
0925 14:45:17.002528 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.002530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.002897 (+ 367us) server_negotiation.cc:407] Connection header received
0925 14:45:17.003709 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.003711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.003727 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.003803 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.004649 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005422 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007051 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007343 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007387 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.011725 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.011726 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.011726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.011734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.013510 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.013958 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.013959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.013960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.014017 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.014241 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.014242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.014242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.014392 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.015496 (+ 1104us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.015539 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.015620 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:45:17.015707 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:42293 (local address 127.21.205.252:37035)
0925 14:45:17.002876 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.002878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.003809 (+ 931us) server_negotiation.cc:407] Connection header received
0925 14:45:17.004268 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.004270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.004281 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.004317 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.004706 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005434 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007051 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007348 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007387 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.012066 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.012067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.012067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.012074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.013811 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.014813 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.014815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.014815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.014893 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.015123 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.015125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.015125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.015258 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.015318 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.015540 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.015621 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 14:45:17.016028 29109 ts_manager.cc:194] Registered new tserver with Master: ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
I20260925 14:45:17.016053 29622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.002729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.21.205.194:42293)
0925 14:45:17.003732 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.003742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.004158 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.004384 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.004385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.004549 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.004610 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.004611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.005575 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.005576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.006946 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.006947 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.007481 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.007482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.007620 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.008269 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.008279 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.011944 (+ 3665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.013903 (+ 1959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.013904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.013904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.014719 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.014980 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.014981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.014981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.015042 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.015366 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.015367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.015406 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.015855 (+ 449us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.015962 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":985,"mutex_wait_us":71,"thread_start_us":61,"threads_started":1}
I20260925 14:45:17.016080 29617 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:17.016255 29617 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.016319 29617 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:17.016495 29616 heartbeater.cc:347] Connected to a master server at 127.21.205.252:37035
I20260925 14:45:17.016541 29616 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.016601 29616 heartbeater.cc:511] Master 127.21.205.252:37035 requested a full tablet report, sending...
I20260925 14:45:17.016644 28500 ts_manager.cc:194] Re-registered known tserver with Master: ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
I20260925 14:45:17.016883 29298 ts_manager.cc:194] Registered new tserver with Master: ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
I20260925 14:45:17.017212 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:42507
W20260925 14:45:17.093719 29623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:17.093919 29623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:17.093957 29623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:17.095070 29623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:17.095130 29623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:17.095175 29623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:17.096684 29623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:33139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:36337
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=38927
--tserver_master_addrs=127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:17.096935 29623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:17.097159 29623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:17.101864 29631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:17.101871 29629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:17.101908 29623 server_base.cc:1034] running on GCE node
W20260925 14:45:17.101874 29628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:17.102200 29623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:17.102460 29623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:17.103595 29623 hybrid_clock.cc:648] HybridClock initialized: now 1790347517103579 us; error 31 us; skew 500 ppm
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347517, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:17.105912 29623 init.cc:377] Logged in from keytab as oryx/127.21.205.195@KRBTEST.COM (short username oryx)
I20260925 14:45:17.106281 29623 webserver.cc:460] Webserver started at http://127.21.205.195:38927/ using document root <none> and password file <none>
I20260925 14:45:17.106449 29623 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:17.106495 29623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:17.107743 29623 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:17.108266 29638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:17.108438 29623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:17.108500 29623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "d6d432e02e2947c9b90b2360cd835983"
format_stamp: "Formatted at 2026-09-25 14:45:16 on dist-test-slave-d7lw"
server_key: "dff9a05418f0f2b4c5a163459401e638"
server_key_iv: "897ba8bbefef1e9be865796e69c1ba5a"
server_key_version: "encryptionkey@0"
I20260925 14:45:17.108825 29623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:17.122231 29623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:17.123878 29623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:17.124010 29623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:17.124224 29623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:17.124511 29623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:17.124567 29623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:17.124599 29623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:17.124626 29623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:17.130412 29623 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36337
I20260925 14:45:17.130443 29753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36337 every 8 connection(s)
I20260925 14:45:17.130759 29623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:45:17.137277 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 29623
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347517, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.252@KRBTEST.COM
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347517, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.253@KRBTEST.COM
Sep 25 14:45:17 dist-test-slave-d7lw krb5kdc[28454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347517, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:17.142988 29161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:39107 (local address 127.21.205.253:34557)
0925 14:45:17.131771 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.131773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.131801 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:45:17.132393 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.132394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.132407 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.132439 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.133273 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.134026 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135121 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.135404 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135471 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.139390 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.139392 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.139393 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.139400 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.141215 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141843 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.141845 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.141845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.141892 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.142257 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.142258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.142258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.142407 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.142452 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.142820 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.142900 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 14:45:17.143169 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:37035 (local address 127.21.205.195:44211)
0925 14:45:17.131600 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.131658 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.132247 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.132515 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.132518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.132938 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.133149 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.133155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.133870 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.133870 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.135039 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.135041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.135496 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135496 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.135644 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.136175 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.136184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.139320 (+ 3136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.141444 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.141445 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.141757 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.141961 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141961 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.141962 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.142156 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.142492 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.142498 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.142649 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.142774 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.142931 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":14,"thread_start_us":96,"threads_started":1}
I20260925 14:45:17.143388 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:60803)
0925 14:45:17.131973 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.131980 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.132260 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.132514 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.132518 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.132933 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.133148 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.133152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.135137 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135137 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.135842 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.135845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.136372 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.136373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.136440 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.136821 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.136826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.139648 (+ 2822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.141955 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.141956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.142213 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.142426 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.142427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.142427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.142478 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.142751 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.142751 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.142797 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.143145 (+ 348us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.143255 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":15,"thread_start_us":50,"threads_started":1}
I20260925 14:45:17.143599 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:34557 (local address 127.21.205.195:39107)
0925 14:45:17.131739 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.131748 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.132247 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.132515 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.132518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.132989 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:17.133149 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.133155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.134110 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.134111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.135039 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.135041 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.135855 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.135933 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.136315 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:17.136319 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:17.139276 (+ 2957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:17.141308 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.141317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.141757 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.141962 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141962 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:17.141962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:17.142156 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:17.142498 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:17.142498 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:17.142649 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.143161 (+ 512us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.143357 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":33,"spinlock_wait_cycles":6528,"thread_start_us":116,"threads_started":1}
I20260925 14:45:17.143633 29484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:44211 (local address 127.21.205.252:37035)
0925 14:45:17.131909 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.131911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.131918 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:17.132394 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.132396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.132407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.132439 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.133270 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.133776 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135133 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.135411 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135471 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.139405 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.139407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.139407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.139414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.141369 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.141837 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.141838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.141838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.141892 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.142251 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.142252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.142252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.142407 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.142465 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.143453 (+ 988us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.143542 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:17.143886 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.131853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:60803 (local address 127.21.205.254:33533)
0925 14:45:17.131924 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.131926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.132033 (+ 107us) server_negotiation.cc:407] Connection header received
0925 14:45:17.132356 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.132357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.132370 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.132453 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.133246 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.134026 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.135942 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.136249 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.136319 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.139728 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.139729 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.139729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.139736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.141870 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.142283 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.142284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.142284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.142337 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.142541 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.142542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.142542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.142666 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.142723 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.143685 (+ 962us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.143770 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:45:17.144289 29756 heartbeater.cc:347] Connected to a master server at 127.21.205.253:34557
I20260925 14:45:17.144384 29756 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.144578 29756 heartbeater.cc:511] Master 127.21.205.253:34557 requested a full tablet report, sending...
I20260925 14:45:17.144613 29755 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33533
I20260925 14:45:17.144644 29755 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.144698 29755 heartbeater.cc:511] Master 127.21.205.254:33533 requested a full tablet report, sending...
I20260925 14:45:17.144748 29754 heartbeater.cc:347] Connected to a master server at 127.21.205.252:37035
I20260925 14:45:17.144781 29754 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:17.144842 29754 heartbeater.cc:511] Master 127.21.205.252:37035 requested a full tablet report, sending...
I20260925 14:45:17.144995 29109 ts_manager.cc:194] Registered new tserver with Master: d6d432e02e2947c9b90b2360cd835983 (127.21.205.195:36337)
I20260925 14:45:17.145061 28500 ts_manager.cc:194] Re-registered known tserver with Master: d6d432e02e2947c9b90b2360cd835983 (127.21.205.195:36337)
I20260925 14:45:17.145679 29298 ts_manager.cc:194] Registered new tserver with Master: d6d432e02e2947c9b90b2360cd835983 (127.21.205.195:36337)
I20260925 14:45:17.145921 28500 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:60803
I20260925 14:45:17.147567 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.138966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.21.205.252:37035)
0925 14:45:17.139100 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.139101 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.139361 (+ 260us) server_negotiation.cc:407] Connection header received
0925 14:45:17.139382 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.139382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.139394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.139451 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.140335 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.141100 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.142165 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.142530 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.142587 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.144113 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.144114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.144115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.144121 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.146159 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.146509 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.146509 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.146510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.146567 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.146839 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.146840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.146841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.146985 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.147084 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.147429 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.147506 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260925 14:45:17.150970 29767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.138493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.21.205.254:33533)
0925 14:45:17.139871 (+ 1378us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.139872 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.139879 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:17.139903 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.139903 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.139914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.144475 (+ 4561us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.144513 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.145302 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.146379 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.146705 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.146759 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.148134 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.148136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.148136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.148143 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.149755 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.150138 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.150139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.150139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.150199 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.150408 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.150409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.150410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.150550 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.150659 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.150788 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.150870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1353,"thread_start_us":72,"threads_started":1}
I20260925 14:45:17.153630 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.138774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.21.205.253:34557)
0925 14:45:17.139207 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.139209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.139356 (+ 147us) server_negotiation.cc:407] Connection header received
0925 14:45:17.139380 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.139381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.139393 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.139450 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.147180 (+ 7730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.148010 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.149017 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.149242 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.149290 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.151077 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.151078 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.151078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.151086 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.152635 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.152980 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.152981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.152981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.153036 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.153236 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.153237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.153237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.153324 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.153392 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.153515 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.153571 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":42,"threads_started":1}
I20260925 14:45:17.152570 28500 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:17.154727 28500 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:17.161861 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.157718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36337 (local address 127.0.0.1:53840)
0925 14:45:17.157780 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.157791 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.157832 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.158304 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.158304 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.158306 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:17.158350 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.158352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.159462 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.159462 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.160685 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.160708 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.161621 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.161623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.161682 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.161685 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.161716 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.161753 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":39}
I20260925 14:45:17.161960 29775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.157740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46105 (local address 127.0.0.1:35602)
0925 14:45:17.157871 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.157882 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.157931 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.158513 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.158514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.158516 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:17.158559 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.158561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.159430 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.159432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.160688 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.160689 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.161717 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.161718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.161766 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.161769 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.161823 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.161853 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":89,"thread_start_us":90,"threads_started":1}
I20260925 14:45:17.162032 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.158146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35602 (local address 127.21.205.193:46105)
0925 14:45:17.158301 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.158305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.158333 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:45:17.158374 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.158376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.158391 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.158477 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:17.158628 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.159333 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.160789 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.161522 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.161625 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.161684 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.161882 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.161928 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260925 14:45:17.162313 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.157970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.21.205.195:36337)
0925 14:45:17.158123 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.158127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.158138 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:45:17.158176 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.158178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.158196 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.158428 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:17.158479 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.159407 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.160819 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.161522 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.162080 (+ 558us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.162172 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.162200 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.162238 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
I20260925 14:45:17.162774 29412 tablet_service.cc:1467] Processing CreateTablet for tablet 0b754fe31dba4077abddf1c01759e457 (DEFAULT_TABLE table=test-table [id=c9c0e123d7a24e5dac106979d36caae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:17.162878 29686 tablet_service.cc:1467] Processing CreateTablet for tablet 0b754fe31dba4077abddf1c01759e457 (DEFAULT_TABLE table=test-table [id=c9c0e123d7a24e5dac106979d36caae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:17.163174 29686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b754fe31dba4077abddf1c01759e457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:17.163192 29412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b754fe31dba4077abddf1c01759e457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:17.165962 29780 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Bootstrap starting.
I20260925 14:45:17.166507 29780 tablet_bootstrap.cc:654] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:17.166678 29776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.157777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:34351 (local address 127.0.0.1:58612)
0925 14:45:17.158412 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.158422 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.158463 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.159678 (+ 1215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.159679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.159681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:17.159727 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.159730 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.160786 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.160787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.161472 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.161474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.166444 (+ 4970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.166445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.166506 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.166510 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.166571 (+ 61us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.166613 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":77,"threads_started":1}
I20260925 14:45:17.166698 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.157982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58612 (local address 127.21.205.194:34351)
0925 14:45:17.159535 (+ 1553us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.159539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.159546 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:17.159576 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.159577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.159590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.159679 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:17.159868 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.160722 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.161564 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.166359 (+ 4795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.166462 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.166522 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.166587 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.166624 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1520,"thread_start_us":80,"threads_started":1}
I20260925 14:45:17.166862 29780 log.cc:824] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:17.167157 29781 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Bootstrap starting.
I20260925 14:45:17.167330 29550 tablet_service.cc:1467] Processing CreateTablet for tablet 0b754fe31dba4077abddf1c01759e457 (DEFAULT_TABLE table=test-table [id=c9c0e123d7a24e5dac106979d36caae8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:17.167629 29780 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: No bootstrap required, opened a new log
I20260925 14:45:17.167644 29550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b754fe31dba4077abddf1c01759e457. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:17.167704 29780 ts_tablet_manager.cc:1397] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:17.167766 29781 tablet_bootstrap.cc:654] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:17.168057 29781 log.cc:824] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:17.168756 29781 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: No bootstrap required, opened a new log
I20260925 14:45:17.168818 29781 ts_tablet_manager.cc:1397] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:17.170313 29780 raft_consensus.cc:348] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.170547 29780 raft_consensus.cc:374] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:17.170594 29780 raft_consensus.cc:729] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72014da9fc04587aec85ceb25d0645f, State: Initialized, Role: FOLLOWER
I20260925 14:45:17.170706 29780 consensus_queue.cc:260] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.171015 29479 heartbeater.cc:503] Master 127.21.205.254:33533 was elected leader, sending a full tablet report...
I20260925 14:45:17.171100 29781 raft_consensus.cc:348] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.171249 29781 raft_consensus.cc:374] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:17.171294 29781 raft_consensus.cc:729] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6d432e02e2947c9b90b2360cd835983, State: Initialized, Role: FOLLOWER
I20260925 14:45:17.171401 29781 consensus_queue.cc:260] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.171671 29781 ts_tablet_manager.cc:1428] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:17.171792 29780 ts_tablet_manager.cc:1428] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 14:45:17.171983 29755 heartbeater.cc:503] Master 127.21.205.254:33533 was elected leader, sending a full tablet report...
I20260925 14:45:17.173066 29785 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Bootstrap starting.
I20260925 14:45:17.173727 29785 tablet_bootstrap.cc:654] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:17.173996 29785 log.cc:824] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:17.174610 29785 tablet_bootstrap.cc:492] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: No bootstrap required, opened a new log
I20260925 14:45:17.174685 29785 ts_tablet_manager.cc:1397] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:17.176438 29785 raft_consensus.cc:348] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.176569 29785 raft_consensus.cc:374] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:17.176611 29785 raft_consensus.cc:729] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ede93d07442948a4aca5b0ef07486820, State: Initialized, Role: FOLLOWER
I20260925 14:45:17.176715 29785 consensus_queue.cc:260] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.176940 29785 ts_tablet_manager.cc:1428] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 14:45:17.177006 29617 heartbeater.cc:503] Master 127.21.205.254:33533 was elected leader, sending a full tablet report...
I20260925 14:45:17.228547 29786 raft_consensus.cc:482] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:17.228642 29786 raft_consensus.cc:504] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.229002 29786 leader_election.cc:288] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105), ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
I20260925 14:45:17.231917 29777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.229245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:40185 (local address 127.21.205.194:34351)
0925 14:45:17.229306 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.229308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.229334 (+ 26us) server_negotiation.cc:407] Connection header received
0925 14:45:17.229357 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.229358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.229366 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.229424 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:17.229583 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.230223 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231163 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.231519 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.231603 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.231775 (+ 172us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.231847 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:45:17.231950 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.229158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:34351 (local address 127.21.205.195:40185)
0925 14:45:17.229215 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.229227 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.229297 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.229427 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.229428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.229429 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:17.229502 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.229504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.230306 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.230307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.231062 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.231064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.231593 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.231681 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.231684 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.231767 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.231842 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 14:45:17.231950 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.229254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:59241 (local address 127.21.205.193:46105)
0925 14:45:17.229298 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.229300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.229381 (+ 81us) server_negotiation.cc:407] Connection header received
0925 14:45:17.229446 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.229447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.229457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.229529 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:17.229670 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.230255 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231156 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.231520 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231614 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.231622 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.231842 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.231878 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:17.231971 29760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.229158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:46105 (local address 127.21.205.195:59241)
0925 14:45:17.229315 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:17.229325 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:17.229391 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:17.229534 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:17.229535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:17.229538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:17.229603 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.229604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.230340 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.230341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.231060 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:17.231061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.231615 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.231616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:17.231733 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.231736 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:17.231837 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 14:45:17.231873 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":115}
I20260925 14:45:17.232254 29570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b754fe31dba4077abddf1c01759e457" candidate_uuid: "d6d432e02e2947c9b90b2360cd835983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede93d07442948a4aca5b0ef07486820" is_pre_election: true
I20260925 14:45:17.232307 29432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b754fe31dba4077abddf1c01759e457" candidate_uuid: "d6d432e02e2947c9b90b2360cd835983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72014da9fc04587aec85ceb25d0645f" is_pre_election: true
I20260925 14:45:17.232403 29570 raft_consensus.cc:2454] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6d432e02e2947c9b90b2360cd835983 in term 0.
I20260925 14:45:17.232435 29432 raft_consensus.cc:2454] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d6d432e02e2947c9b90b2360cd835983 in term 0.
I20260925 14:45:17.232599 29642 leader_election.cc:302] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6d432e02e2947c9b90b2360cd835983, ede93d07442948a4aca5b0ef07486820; no voters:
I20260925 14:45:17.232764 29786 raft_consensus.cc:2781] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:17.232807 29786 raft_consensus.cc:482] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:17.232823 29786 raft_consensus.cc:3037] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:17.233486 29786 raft_consensus.cc:504] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.233618 29786 leader_election.cc:288] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [CANDIDATE]: Term 1 election: Requested vote from peers e72014da9fc04587aec85ceb25d0645f (127.21.205.193:46105), ede93d07442948a4aca5b0ef07486820 (127.21.205.194:34351)
I20260925 14:45:17.233743 29432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b754fe31dba4077abddf1c01759e457" candidate_uuid: "d6d432e02e2947c9b90b2360cd835983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e72014da9fc04587aec85ceb25d0645f"
I20260925 14:45:17.233770 29570 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b754fe31dba4077abddf1c01759e457" candidate_uuid: "d6d432e02e2947c9b90b2360cd835983" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ede93d07442948a4aca5b0ef07486820"
I20260925 14:45:17.233829 29570 raft_consensus.cc:3037] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:17.233824 29432 raft_consensus.cc:3037] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:17.234647 29570 raft_consensus.cc:2454] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6d432e02e2947c9b90b2360cd835983 in term 1.
I20260925 14:45:17.234684 29432 raft_consensus.cc:2454] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6d432e02e2947c9b90b2360cd835983 in term 1.
I20260925 14:45:17.234827 29639 leader_election.cc:302] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d6d432e02e2947c9b90b2360cd835983, e72014da9fc04587aec85ceb25d0645f; no voters:
I20260925 14:45:17.234962 29786 raft_consensus.cc:2781] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:17.235134 29786 raft_consensus.cc:686] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 1 LEADER]: Becoming Leader. State: Replica: d6d432e02e2947c9b90b2360cd835983, State: Running, Role: LEADER
I20260925 14:45:17.235221 29786 consensus_queue.cc:237] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } }
I20260925 14:45:17.235953 28500 catalog_manager.cc:5495] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 reported cstate change: term changed from 0 to 1, leader changed from <none> to d6d432e02e2947c9b90b2360cd835983 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "d6d432e02e2947c9b90b2360cd835983" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6d432e02e2947c9b90b2360cd835983" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 } health_report { overall_health: UNKNOWN } } }
W20260925 14:45:17.252411 29619 tablet.cc:2369] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:17.252796 29778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.250206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.21.205.195:36337)
0925 14:45:17.250244 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.250246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.250254 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:17.250322 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.250323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.250339 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.250415 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:17.250560 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.251143 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.251959 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.252224 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.252293 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.252459 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:17.252597 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:17.252671 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.252693 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.252712 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:45:17.254309 29432 raft_consensus.cc:1261] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 1 FOLLOWER]: Refusing update from remote peer d6d432e02e2947c9b90b2360cd835983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:17.254323 29570 raft_consensus.cc:1261] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 1 FOLLOWER]: Refusing update from remote peer d6d432e02e2947c9b90b2360cd835983: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:17.254616 29789 consensus_queue.cc:1035] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72014da9fc04587aec85ceb25d0645f" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 46105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:17.254755 29786 consensus_queue.cc:1035] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ede93d07442948a4aca5b0ef07486820" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:17.257053 29792 mvcc.cc:205] Tried to move back new op lower bound from 7333263430671527936 to 7333263430595997696. Current Snapshot: MvccSnapshot[applied={T|T < 7333263430671527936}]
I20260925 14:45:17.258196 29794 mvcc.cc:205] Tried to move back new op lower bound from 7333263430671527936 to 7333263430595997696. Current Snapshot: MvccSnapshot[applied={T|T < 7333263430671527936}]
I20260925 14:45:17.259193 28500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:17.259284 28500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:17.259922 29793 mvcc.cc:205] Tried to move back new op lower bound from 7333263430671527936 to 7333263430595997696. Current Snapshot: MvccSnapshot[applied={T|T < 7333263430671527936 or (T in {7333263430671527936})}]
I20260925 14:45:17.261030 28500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e933784179cb47f8b4bf2872e582e17c: Sending DeleteTablet for 3 replicas of tablet 0b754fe31dba4077abddf1c01759e457
I20260925 14:45:17.261340 29412 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b754fe31dba4077abddf1c01759e457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:17 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:35602
I20260925 14:45:17.261343 29550 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b754fe31dba4077abddf1c01759e457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:17 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:58612
I20260925 14:45:17.261569 29686 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b754fe31dba4077abddf1c01759e457 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:17 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:53840
I20260925 14:45:17.261693 29801 tablet_replica.cc:330] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: stopping tablet replica
I20260925 14:45:17.261974 29801 raft_consensus.cc:2229] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:17.262087 29801 raft_consensus.cc:2258] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:17.262213 29802 tablet_replica.cc:330] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: stopping tablet replica
I20260925 14:45:17.262271 29802 raft_consensus.cc:2229] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:17.262333 29802 raft_consensus.cc:2258] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:17.262725 29801 ts_tablet_manager.cc:1905] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:17.262917 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28467
I20260925 14:45:17.264112 29802 ts_tablet_manager.cc:1905] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:17.264679 29801 ts_tablet_manager.cc:1918] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:17.264742 29801 log.cc:1196] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/0b754fe31dba4077abddf1c01759e457
I20260925 14:45:17.265007 29801 ts_tablet_manager.cc:1939] T 0b754fe31dba4077abddf1c01759e457 P ede93d07442948a4aca5b0ef07486820: Deleting consensus metadata
I20260925 14:45:17.265605 29803 tablet_replica.cc:330] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: stopping tablet replica
I20260925 14:45:17.265697 29803 raft_consensus.cc:2229] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:17.265784 29803 raft_consensus.cc:2258] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:17.266367 29803 ts_tablet_manager.cc:1905] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:17.268046 29803 ts_tablet_manager.cc:1918] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:17.268112 29803 log.cc:1196] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/0b754fe31dba4077abddf1c01759e457
I20260925 14:45:17.268388 29803 ts_tablet_manager.cc:1939] T 0b754fe31dba4077abddf1c01759e457 P d6d432e02e2947c9b90b2360cd835983: Deleting consensus metadata
W20260925 14:45:17.271538 29505 connection.cc:535] server connection from 127.0.0.1:58612 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 14:45:17.271816 29641 connection.cc:535] server connection from 127.0.0.1:53840 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 14:45:17.273404 29802 ts_tablet_manager.cc:1918] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:17.273481 29802 log.cc:1196] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/0b754fe31dba4077abddf1c01759e457
I20260925 14:45:17.273762 29802 ts_tablet_manager.cc:1939] T 0b754fe31dba4077abddf1c01759e457 P e72014da9fc04587aec85ceb25d0645f: Deleting consensus metadata
I20260925 14:45:17.273836 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.261875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.21.205.252:37035)
0925 14:45:17.261911 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.261913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.265966 (+ 4053us) server_negotiation.cc:407] Connection header received
0925 14:45:17.266052 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.266053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.266066 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.266111 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.266938 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.267677 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.268915 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.269165 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.269250 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.270499 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.270501 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.270501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.270519 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.272293 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.272705 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.272706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.272706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.272761 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.273032 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.273033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.273033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.273166 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.273297 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.273624 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.273723 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 14:45:17.274058 29367 connection.cc:441] server connection from 127.0.0.1:35602 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35602 (request call id 1) could send its response
I20260925 14:45:17.281414 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.273467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.21.205.253:34557)
0925 14:45:17.273505 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.273507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.273721 (+ 214us) server_negotiation.cc:407] Connection header received
0925 14:45:17.273810 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.273811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.273824 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.273853 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.274576 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.275323 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.276398 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.276732 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.276804 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.278232 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.278233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.278234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.278240 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.279922 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.280274 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.280275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.280275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.280331 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.280885 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.280886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.280886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.281010 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.281113 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.281241 (+ 128us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.281321 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:17.282558 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:17.274836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.21.205.252:37035)
0925 14:45:17.274870 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:17.274872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:17.275000 (+ 128us) server_negotiation.cc:407] Connection header received
0925 14:45:17.275068 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:17.275069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:17.275077 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:17.275125 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:17.275725 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.276419 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.277601 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:17.277950 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:17.277995 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:17.279576 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:17.279577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:17.279578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:17.279584 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:17.281427 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.281883 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.281884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.281885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.281942 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:17.282143 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:17.282144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:17.282144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:17.282261 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:17.282350 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:17.282410 (+ 60us) server_negotiation.cc:299] Negotiation successful
0925 14:45:17.282477 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:45:18.258733 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:18.258470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:55745)
0925 14:45:18.258630 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:18.258692 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":100,"threads_started":1}
I20260925 14:45:18.258725 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:18.258475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:42599)
0925 14:45:18.258605 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:18.258681 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
W20260925 14:45:18.258946 29755 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:33533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
W20260925 14:45:18.258949 29479 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:33533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:18.262226 29814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:18.262027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:45409)
0925 14:45:18.262128 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:18.262188 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":49,"threads_started":1}
W20260925 14:45:18.262440 29617 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:33533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:18.761878 29815 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:18.762012 29815 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:18.762219 29815 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:18.762563 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:18.762378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43550)
0925 14:45:18.762472 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:18.762527 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76,"thread_start_us":58,"threads_started":1}
W20260925 14:45:18.762708 29096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:18.763108 29816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:18.762988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43552)
0925 14:45:18.763041 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:18.763086 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 14:45:18.763201 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:18.763281 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:18.763393 29815 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:19.259605 29818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:19.259355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:46255)
0925 14:45:19.259502 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:19.259557 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260925 14:45:19.259635 29819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:19.259443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:33189)
0925 14:45:19.259562 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:19.259596 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260925 14:45:19.262895 29820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:19.262723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:38713)
0925 14:45:19.262838 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:19.262860 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":58,"threads_started":1}
I20260925 14:45:20.260469 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:20.260221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:59661)
0925 14:45:20.260347 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:20.260392 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":75,"threads_started":1}
I20260925 14:45:20.260489 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:20.260221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:53727)
0925 14:45:20.260347 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:20.260390 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260925 14:45:20.263445 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:20.263288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:40997)
0925 14:45:20.263382 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:20.263412 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":47,"threads_started":1}
I20260925 14:45:20.884858 29824 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:20.884943 29824 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:20.885075 29824 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:20.885454 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:20.885252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43860)
0925 14:45:20.885362 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:20.885419 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":70,"threads_started":1}
I20260925 14:45:20.885836 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:20.885752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:43862)
0925 14:45:20.885796 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:20.885820 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 14:45:20.885921 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:20.885959 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:20.886040 29824 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:21.261257 29828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:21.261039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:60347)
0925 14:45:21.261147 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:21.261207 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":61,"threads_started":1}
I20260925 14:45:21.261236 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:21.260992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:35889)
0925 14:45:21.261135 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:21.261178 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260925 14:45:21.263993 29829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:21.263822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:59631)
0925 14:45:21.263923 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:21.263954 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":51,"threads_started":1}
I20260925 14:45:22.262032 29830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:22.261784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:49529)
0925 14:45:22.261925 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:22.261969 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260925 14:45:22.262056 29831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:22.261810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:60637)
0925 14:45:22.261954 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:22.262010 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260925 14:45:22.264524 29832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:22.264355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:33591)
0925 14:45:22.264458 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:22.264490 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":54,"threads_started":1}
I20260925 14:45:22.821271 29833 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:22.821382 29833 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:22.821527 29833 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:22.821906 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:22.821708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:44050)
0925 14:45:22.821826 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:22.821869 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260925 14:45:22.822342 29834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:22.822214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:44062)
0925 14:45:22.822268 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:22.822301 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:45:22.822433 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:22.822490 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:22.822573 29833 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:23.262812 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:23.262619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:54533)
0925 14:45:23.262731 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:23.262767 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":58,"threads_started":1}
I20260925 14:45:23.262964 29836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:23.262504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:40789)
0925 14:45:23.262869 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:23.262911 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":112,"threads_started":1}
I20260925 14:45:23.265089 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:23.264887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:57619)
0925 14:45:23.265008 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:23.265048 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260925 14:45:24.263664 29840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:24.263346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:53941)
0925 14:45:24.263543 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:24.263607 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":103,"threads_started":1}
I20260925 14:45:24.263665 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:24.263358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:36843)
0925 14:45:24.263520 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:24.263595 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260925 14:45:24.265679 29841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:24.265548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:53975)
0925 14:45:24.265625 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:24.265648 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66,"thread_start_us":37,"threads_started":1}
I20260925 14:45:25.136389 29842 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:25.136468 29842 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:25.136596 29842 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:25.136907 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:25.136737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:44248)
0925 14:45:25.136837 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:25.136875 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82,"thread_start_us":58,"threads_started":1}
W20260925 14:45:25.136992 29096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260925 14:45:25.137405 29843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:25.137267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:44252)
0925 14:45:25.137316 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:25.137360 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 14:45:25.137494 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:25.137568 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:25.137639 29842 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:25.264410 29845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:25.264123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:40803)
0925 14:45:25.264283 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:25.264332 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":92,"threads_started":1}
I20260925 14:45:25.264637 29846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:25.264388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:41933)
0925 14:45:25.264550 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:25.264586 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":80,"threads_started":1}
I20260925 14:45:25.266247 29847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:25.266076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:50993)
0925 14:45:25.266184 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:25.266215 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":56,"threads_started":1}
I20260925 14:45:26.265162 29848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:26.264911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:44441)
0925 14:45:26.265046 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:26.265102 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260925 14:45:26.265702 29849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:26.265442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:53981)
0925 14:45:26.265618 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:26.265656 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":50,"threads_started":1}
I20260925 14:45:26.266775 29850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:26.266606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:35377)
0925 14:45:26.266707 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:26.266742 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260925 14:45:27.266573 29851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:27.265934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:43805)
0925 14:45:27.266288 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:27.266334 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":66,"threads_started":1}
I20260925 14:45:27.266651 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:27.266462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:35225)
0925 14:45:27.266555 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:27.266603 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75,"thread_start_us":49,"threads_started":1}
I20260925 14:45:27.267269 29853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:27.267097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:42639)
0925 14:45:27.267207 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:27.267238 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
I20260925 14:45:27.689989 29854 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:27.690104 29854 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:27.690229 29854 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:27.690630 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:27.690440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:48426)
0925 14:45:27.690553 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:27.690592 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260925 14:45:27.691045 29855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:27.690934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:48430)
0925 14:45:27.690988 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:27.691024 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 14:45:27.691156 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:27.691214 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:27.691299 29854 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:28.267302 29857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:28.267031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:49855)
0925 14:45:28.267181 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:28.267246 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260925 14:45:28.267452 29858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:28.267189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:52005)
0925 14:45:28.267336 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:28.267398 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":64,"threads_started":1}
I20260925 14:45:28.267815 29859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:28.267644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:40171)
0925 14:45:28.267722 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:28.267745 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67,"thread_start_us":38,"threads_started":1}
I20260925 14:45:29.267985 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:29.267717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:40685)
0925 14:45:29.267886 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:29.267930 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260925 14:45:29.268091 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:29.267875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:56269)
0925 14:45:29.267975 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:29.268039 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260925 14:45:29.268352 29862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:29.268163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:51291)
0925 14:45:29.268282 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:29.268320 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260925 14:45:30.268805 29863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:30.268582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:55111)
0925 14:45:30.268709 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:30.268748 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":79,"threads_started":1}
I20260925 14:45:30.268880 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:30.268634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:44635)
0925 14:45:30.268806 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:30.268839 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
I20260925 14:45:30.269074 29865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:30.268885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:39739)
0925 14:45:30.268989 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:30.269024 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":46,"threads_started":1}
I20260925 14:45:31.269456 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:31.269246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:41869)
0925 14:45:31.269365 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:31.269400 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260925 14:45:31.269532 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:31.269257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:49405)
0925 14:45:31.269426 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:31.269486 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260925 14:45:31.269596 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:31.269413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:60821)
0925 14:45:31.269520 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:31.269554 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":53,"threads_started":1}
I20260925 14:45:32.270212 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:32.269966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:59695)
0925 14:45:32.270114 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:32.270156 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
I20260925 14:45:32.270243 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:32.269966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:58813)
0925 14:45:32.270152 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:32.270192 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":63,"threads_started":1}
I20260925 14:45:32.270362 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:32.270059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:43211)
0925 14:45:32.270267 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:32.270302 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":57,"threads_started":1}
I20260925 14:45:33.270871 29873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:33.270648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:38845)
0925 14:45:33.270781 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:33.270816 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":56,"threads_started":1}
I20260925 14:45:33.270961 29872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:33.270627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:39173)
0925 14:45:33.270898 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:33.270927 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":77,"threads_started":1}
I20260925 14:45:33.270961 29874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:33.270772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:58497)
0925 14:45:33.270881 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:33.270916 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260925 14:45:34.271552 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:34.271300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:56633)
0925 14:45:34.271428 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:34.271474 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260925 14:45:34.271615 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:34.271369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:34285)
0925 14:45:34.271468 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:34.271509 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":42,"threads_started":1}
I20260925 14:45:34.271840 29877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:34.271557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:47893)
0925 14:45:34.271741 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:34.271803 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":65,"threads_started":1}
I20260925 14:45:35.272184 29878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:35.271949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:60787)
0925 14:45:35.272088 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:35.272123 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":58,"threads_started":1}
I20260925 14:45:35.272294 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:35.272083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:39483)
0925 14:45:35.272187 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:35.272222 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
I20260925 14:45:35.272491 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:35.272201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:41725)
0925 14:45:35.272419 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:35.272455 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":46,"threads_started":1}
I20260925 14:45:36.272787 29881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:36.272538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:59985)
0925 14:45:36.272665 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:36.272729 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":68,"threads_started":1}
I20260925 14:45:36.272941 29882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:36.272657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:54313)
0925 14:45:36.272847 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:36.272885 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":83,"threads_started":1}
I20260925 14:45:36.273164 29883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:36.272881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:37327)
0925 14:45:36.273065 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:36.273100 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":53,"threads_started":1}
I20260925 14:45:37.273599 29884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:37.273223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:57277)
0925 14:45:37.273498 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:37.273541 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":63,"threads_started":1}
I20260925 14:45:37.273629 29885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:37.273350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:58629)
0925 14:45:37.273487 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:37.273531 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260925 14:45:37.273782 29886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:37.273611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:50403)
0925 14:45:37.273707 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:37.273738 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":44,"threads_started":1}
I20260925 14:45:38.117600 29887 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e933784179cb47f8b4bf2872e582e17c)
I20260925 14:45:38.117717 29887 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "e933784179cb47f8b4bf2872e582e17c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33533 } } peers { permanent_uuid: "f7da9f120f0c46a2bdceedd36d832bff" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 34557 } attrs { promote: false } } peers { permanent_uuid: "eff2d92b23c34955b614c82235da890f" member_type: NON_VOTER last_known_addr { host: "127.21.205.252" port: 37035 } attrs { promote: true } }
I20260925 14:45:38.118052 29887 leader_election.cc:288] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533)
I20260925 14:45:38.118424 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:38.118238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:38836)
0925 14:45:38.118342 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:38.118385 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":63,"threads_started":1}
W20260925 14:45:38.118508 29096 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260925 14:45:38.118891 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:38.118773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.0.0.1:38848)
0925 14:45:38.118833 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:38.118867 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 14:45:38.118968 29096 leader_election.cc:334] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e933784179cb47f8b4bf2872e582e17c (127.21.205.254:33533): Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
I20260925 14:45:38.119011 29096 leader_election.cc:302] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: f7da9f120f0c46a2bdceedd36d832bff; no voters: e933784179cb47f8b4bf2872e582e17c
I20260925 14:45:38.119086 29887 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f7da9f120f0c46a2bdceedd36d832bff [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:45:38.274263 29892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:38.274045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:45923)
0925 14:45:38.274173 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:38.274218 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":50,"threads_started":1}
I20260925 14:45:38.274281 29891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:38.274008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:55113)
0925 14:45:38.274193 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:38.274234 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260925 14:45:38.274302 29893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:38.274122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:52173)
0925 14:45:38.274227 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:38.274252 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":55,"threads_started":1}
I20260925 14:45:39.274904 29895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:39.274678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:59787)
0925 14:45:39.274805 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:39.274844 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":56,"threads_started":1}
I20260925 14:45:39.275003 29896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:39.274733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:34747)
0925 14:45:39.274916 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:39.274955 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":49,"threads_started":1}
I20260925 14:45:39.275030 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:39.274650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:56779)
0925 14:45:39.274961 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:39.274990 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":61,"threads_started":1}
I20260925 14:45:40.275700 29898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:40.275485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:45053)
0925 14:45:40.275590 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:40.275644 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":42,"threads_started":1}
I20260925 14:45:40.275698 29897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:40.275357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:43731)
0925 14:45:40.275565 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:40.275634 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":70,"threads_started":1}
I20260925 14:45:40.275885 29899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:40.275653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:47165)
0925 14:45:40.275742 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:40.275769 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74,"thread_start_us":40,"threads_started":1}
I20260925 14:45:41.276494 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:41.276326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:54867)
0925 14:45:41.276432 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:41.276458 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":46,"threads_started":1}
I20260925 14:45:41.276486 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:41.276252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:60887)
0925 14:45:41.276388 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:41.276431 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260925 14:45:41.276796 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:41.276399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:44775)
0925 14:45:41.276706 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:41.276744 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":55,"threads_started":1}
I20260925 14:45:42.277290 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:42.277059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:40091)
0925 14:45:42.277188 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:42.277236 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":72,"threads_started":1}
I20260925 14:45:42.277290 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:42.277070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:38191)
0925 14:45:42.277218 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:42.277244 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":62,"threads_started":1}
I20260925 14:45:42.277541 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:42.277281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:55093)
0925 14:45:42.277451 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:42.277490 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":45,"threads_started":1}
I20260925 14:45:43.278227 29906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:43.277931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:60821)
0925 14:45:43.278121 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:43.278166 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
I20260925 14:45:43.278281 29907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:43.277969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:33249)
0925 14:45:43.278188 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:43.278229 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":46,"threads_started":1}
I20260925 14:45:43.278484 29908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:43.278005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:34189)
0925 14:45:43.278410 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:43.278444 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":66,"threads_started":1}
I20260925 14:45:44.279105 29910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:44.278868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:48695)
0925 14:45:44.279004 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:44.279046 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
I20260925 14:45:44.279311 29909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:44.278836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:49955)
0925 14:45:44.279209 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:44.279247 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":67,"threads_started":1}
I20260925 14:45:44.279484 29911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:44.279021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:54163)
0925 14:45:44.279393 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:44.279438 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":65,"threads_started":1}
I20260925 14:45:45.280004 29912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:45.279617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:46225)
0925 14:45:45.279896 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:45.279940 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":70,"threads_started":1}
I20260925 14:45:45.280097 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:45.279733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:42843)
0925 14:45:45.280003 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:45.280045 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":92,"threads_started":1}
I20260925 14:45:45.280288 29914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:45.279968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:44251)
0925 14:45:45.280202 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:45.280241 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":46,"threads_started":1}
I20260925 14:45:46.280838 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:46.280613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:38545)
0925 14:45:46.280737 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:46.280778 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":43,"threads_started":1}
I20260925 14:45:46.280933 29915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:46.280501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:34721)
0925 14:45:46.280835 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:46.280878 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":68,"threads_started":1}
I20260925 14:45:46.281157 29917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:46.280692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:57635)
0925 14:45:46.281079 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:46.281123 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":58,"threads_started":1}
I20260925 14:45:47.281592 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:47.281332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.193:44595)
0925 14:45:47.281491 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:47.281530 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":79,"threads_started":1}
I20260925 14:45:47.281664 29918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:47.281332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.194:41611)
0925 14:45:47.281511 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:47.281557 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260925 14:45:47.282047 29920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:47.281539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33533 (local address 127.21.205.195:35091)
0925 14:45:47.281943 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:47.281983 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:33533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":59,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:903: Failure
Failed
Bad status: Timed out: Could not connect to the cluster: ConnectToClusterRpc(addrs: 127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035, num_attempts: 243) passed its deadline: Not found: no leader found: ConnectToClusterRpc(addrs: 127.21.205.254:33533,127.21.205.253:34557,127.21.205.252:37035, num_attempts: 1)
I20260925 14:45:47.450897 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 29259
I20260925 14:45:47.457075 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 29486
I20260925 14:45:47.462525 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 29623
I20260925 14:45:47.468714 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 28968
I20260925 14:45:47.473852 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 29164
2026-09-25T14:45:47Z chronyd exiting
I20260925 14:45:47.491549 22327 test_util.cc:175] -----------------------------------------------
I20260925 14:45:47.491611 22327 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790347486435597-22327-0
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (32977 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:47 dist-test-slave-d7lw krb5kdc[29923](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:47 dist-test-slave-d7lw krb5kdc[29923](info): set up 2 sockets
Sep 25 14:45:47 dist-test-slave-d7lw krb5kdc[29923](info): commencing operation
krb5kdc: starting...
W20260925 14:45:48.515522 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:48 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab.
Entry for principal oryx/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab.
I20260925 14:45:48.636303 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33615
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab
--principal=oryx/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:48.721987 29936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:48.722211 29936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:48.722236 29936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:48.723373 29936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:48.723413 29936 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 14:45:48.723426 29936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:48.723439 29936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:48.723452 29936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:48.723464 29936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:48.725037 29936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33615
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.254.keytab
--principal=oryx/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:48.725275 29936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:48.725467 29936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:48.730773 29944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:48.730777 29936 server_base.cc:1034] running on GCE node
W20260925 14:45:48.730759 29942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:48.730756 29941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:48.731321 29936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:48.731669 29936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:48.732841 29936 hybrid_clock.cc:648] HybridClock initialized: now 1790347548732825 us; error 44 us; skew 500 ppm
Sep 25 14:45:48 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:48.740815 29936 init.cc:377] Logged in from keytab as oryx/127.21.205.254@KRBTEST.COM (short username oryx)
I20260925 14:45:48.741369 29936 webserver.cc:460] Webserver started at http://127.21.205.254:44413/ using document root <none> and password file <none>
I20260925 14:45:48.741626 29936 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:48.741695 29936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:48.741844 29936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:48.743005 29936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "d44693497bf045c6829ccf1e7b0649a4"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "e8645b8a5f12f5240119d1eb3150ba53"
server_key_iv: "f5c728bb4d432f3678fbe8a9bfb8f31c"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.743465 29936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "d44693497bf045c6829ccf1e7b0649a4"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "e8645b8a5f12f5240119d1eb3150ba53"
server_key_iv: "f5c728bb4d432f3678fbe8a9bfb8f31c"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.744976 29936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:45:48.745795 29951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:48.745999 29936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:48.746134 29936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "d44693497bf045c6829ccf1e7b0649a4"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "e8645b8a5f12f5240119d1eb3150ba53"
server_key_iv: "f5c728bb4d432f3678fbe8a9bfb8f31c"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.746222 29936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:48.766402 29936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:48.768647 29936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:48.768843 29936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:48.774398 29936 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33615
I20260925 14:45:48.774412 30013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33615 every 8 connection(s)
I20260925 14:45:48.774873 29936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:48.775650 30014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:48.778141 30014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Bootstrap starting.
I20260925 14:45:48.778820 30014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:48.779141 30014 log.cc:824] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:48.779908 30014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: No bootstrap required, opened a new log
I20260925 14:45:48.781827 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 29936
I20260925 14:45:48.781934 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:48.782173 22327 external_mini_cluster.cc:1428] Setting key c24e71a07538df0e2b33fbc11b7a9079
I20260925 14:45:48.782240 30014 raft_consensus.cc:348] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } }
I20260925 14:45:48.782452 30014 raft_consensus.cc:374] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:48.782497 30014 raft_consensus.cc:729] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d44693497bf045c6829ccf1e7b0649a4, State: Initialized, Role: FOLLOWER
I20260925 14:45:48.782680 30014 consensus_queue.cc:260] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } }
I20260925 14:45:48.782780 30014 raft_consensus.cc:388] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:48.782822 30014 raft_consensus.cc:482] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:48.782886 30014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:48.783576 30014 raft_consensus.cc:504] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } }
I20260925 14:45:48.783704 30014 leader_election.cc:302] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d44693497bf045c6829ccf1e7b0649a4; no voters:
I20260925 14:45:48.783910 30014 leader_election.cc:288] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:48.783973 30019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:48.784128 30019 raft_consensus.cc:686] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [term 1 LEADER]: Becoming Leader. State: Replica: d44693497bf045c6829ccf1e7b0649a4, State: Running, Role: LEADER
I20260925 14:45:48.784217 30014 sys_catalog.cc:564] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:48.784227 30019 consensus_queue.cc:237] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } }
I20260925 14:45:48.784658 30019 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d44693497bf045c6829ccf1e7b0649a4. Latest consensus state: current_term: 1 leader_uuid: "d44693497bf045c6829ccf1e7b0649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } } }
I20260925 14:45:48.784762 30019 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:48.784667 30020 sys_catalog.cc:455] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d44693497bf045c6829ccf1e7b0649a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d44693497bf045c6829ccf1e7b0649a4" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33615 } } }
I20260925 14:45:48.784917 30020 sys_catalog.cc:458] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:48.785022 30029 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:48.785624 30029 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:48.787169 30037 mvcc.cc:205] Tried to move back new op lower bound from 7333263559830425600 to 7333263559820828672. Current Snapshot: MvccSnapshot[applied={T|T < 7333263559830425600}]
I20260925 14:45:48.787603 30029 catalog_manager.cc:1347] Generated new cluster ID: 4b793dfe51e543f2a7bc32ecc98a36f3
I20260925 14:45:48.787662 30029 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:48 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:48.793978 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:48.782767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.21.205.254:33615)
0925 14:45:48.783029 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:48.783037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:48.783072 (+ 35us) server_negotiation.cc:407] Connection header received
0925 14:45:48.783188 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:48.783194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:48.783243 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:48.783382 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:48.784281 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.785888 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.786776 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.787433 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.787565 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:48.790011 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:48.790015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:48.790016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:48.790030 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:48.792334 (+ 2304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:48.792713 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:48.792718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:48.792718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:48.792828 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:48.793088 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:48.793089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:48.793089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:48.793355 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:48.793532 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:48.793609 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 14:45:48.793761 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260925 14:45:48.820897 30029 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:48.821492 30029 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:48.828785 30029 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Generated new TSK 0
I20260925 14:45:48.829032 30029 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
I20260925 14:45:48.860855 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33615
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:48.953207 30041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:48.953457 30041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:48.953488 30041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:48.954873 30041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:48.954928 30041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:48.954953 30041 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:45:48.955005 30041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:48.956945 30041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:33615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:48.957229 30041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:48.957517 30041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:48.963028 30046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:48.963016 30047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:48.963016 30049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:48.963048 30041 server_base.cc:1034] running on GCE node
I20260925 14:45:48.963454 30041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:48.963733 30041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:48.964957 30041 hybrid_clock.cc:648] HybridClock initialized: now 1790347548964931 us; error 44 us; skew 500 ppm
Sep 25 14:45:48 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:48.968652 30041 init.cc:377] Logged in from keytab as oryx/127.21.205.193@KRBTEST.COM (short username oryx)
I20260925 14:45:48.969153 30041 webserver.cc:460] Webserver started at http://127.21.205.193:36185/ using document root <none> and password file <none>
I20260925 14:45:48.969413 30041 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:48.969484 30041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:48.969626 30041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:48.970737 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "f3bed538261d4528a6d843c0855d2b96"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "2a079aef3dcc4283e14334d388dd56d0"
server_key_iv: "1c1f2d0c6e3183bfd107aee9b8700190"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.971114 30041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "f3bed538261d4528a6d843c0855d2b96"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "2a079aef3dcc4283e14334d388dd56d0"
server_key_iv: "1c1f2d0c6e3183bfd107aee9b8700190"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.972601 30041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:48.973670 30056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:48.973867 30041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:48.973960 30041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "f3bed538261d4528a6d843c0855d2b96"
format_stamp: "Formatted at 2026-09-25 14:45:48 on dist-test-slave-d7lw"
server_key: "2a079aef3dcc4283e14334d388dd56d0"
server_key_iv: "1c1f2d0c6e3183bfd107aee9b8700190"
server_key_version: "encryptionkey@0"
I20260925 14:45:48.974042 30041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:48.983310 30041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:48.985989 30041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:48.986155 30041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:48.986701 30041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:48.986745 30041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:48.986793 30041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:48.986820 30041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:48.993865 30041 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37669
I20260925 14:45:48.994001 30172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37669 every 8 connection(s)
I20260925 14:45:48.994341 30041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:45:48 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:48.996867 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30041
I20260925 14:45:48.996982 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:48.997203 22327 external_mini_cluster.cc:1428] Setting key 002db0c517e668a9cb691ef9a2f77cfa
I20260925 14:45:48.998519 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:48.987249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:44347 (local address 127.21.205.254:33615)
0925 14:45:48.987329 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:48.987332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:48.987539 (+ 207us) server_negotiation.cc:407] Connection header received
0925 14:45:48.988203 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:48.988205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:48.988218 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:48.988268 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:48.989233 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.990124 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.991903 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.992279 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.992376 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:48.995306 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:48.995307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:48.995308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:48.995322 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:48.996881 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:48.997467 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:48.997468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:48.997468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:48.997535 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:48.997786 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:48.997787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:48.997787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:48.997942 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:48.998016 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:48.998371 (+ 355us) server_negotiation.cc:299] Negotiation successful
0925 14:45:48.998455 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:45:48.998873 30065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:48.987236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.193:44347)
0925 14:45:48.987410 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:48.987459 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:48.988086 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:48.988347 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:48.988351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:48.988776 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:48.989114 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:48.989121 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.990245 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.990246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:48.991723 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:48.991729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:48.992398 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:48.992399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:48.992574 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:48.993320 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:48.993330 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:48.995180 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:48.997021 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:48.997032 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:48.997035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:48.997382 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:48.997604 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:48.997605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:48.997605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:48.997726 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:48.998019 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:48.998026 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:48.998223 (+ 197us) client_negotiation.cc:769] Sending connection context
0925 14:45:48.998350 (+ 127us) client_negotiation.cc:240] Negotiation successful
0925 14:45:48.998552 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260925 14:45:48.999434 30173 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33615
I20260925 14:45:48.999526 30173 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:48.999730 30173 heartbeater.cc:511] Master 127.21.205.254:33615 requested a full tablet report, sending...
I20260925 14:45:49.000420 29967 ts_manager.cc:194] Registered new tserver with Master: f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669)
I20260925 14:45:49.001448 29967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:44347
WARNING: no policy specified for oryx/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.194@KRBTEST.COM" created.
I20260925 14:45:49.010035 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.004162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:36357 (local address 127.21.205.254:33615)
0925 14:45:49.004218 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.004221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.004330 (+ 109us) server_negotiation.cc:407] Connection header received
0925 14:45:49.004412 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.004413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.004431 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.004495 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.004839 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.005336 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.006105 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.006305 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.006390 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.007384 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.007385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.007386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.007394 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.008854 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.009217 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.009218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.009219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.009282 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.009491 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.009492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.009492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.009611 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.009682 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.009841 (+ 159us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.009939 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:45:49.010039 30180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.004090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.193:36357)
0925 14:45:49.004255 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.004271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.004348 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.004501 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.004502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.004703 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.004761 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.004763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.005438 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.005439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.006006 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.006009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.006401 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.006402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.006477 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.006978 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.006984 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.007296 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.008955 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.008957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.008957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.009137 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.009355 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.009357 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.009357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.009425 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.009686 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.009687 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.009735 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.009814 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.009932 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":91,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
I20260925 14:45:49.023204 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33615
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:49.108191 30182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:49.108402 30182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:49.108453 30182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:49.109582 30182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:49.109619 30182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:49.109640 30182 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:45:49.109658 30182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:49.111127 30182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:33615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:49.111325 30182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:49.111516 30182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:49.116737 30188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:49.116715 30187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:49.116746 30182 server_base.cc:1034] running on GCE node
W20260925 14:45:49.116715 30190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:49.117075 30182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:49.117342 30182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:49.118492 30182 hybrid_clock.cc:648] HybridClock initialized: now 1790347549118484 us; error 35 us; skew 500 ppm
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:49.121131 30182 init.cc:377] Logged in from keytab as oryx/127.21.205.194@KRBTEST.COM (short username oryx)
I20260925 14:45:49.121600 30182 webserver.cc:460] Webserver started at http://127.21.205.194:39539/ using document root <none> and password file <none>
I20260925 14:45:49.121836 30182 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:49.121903 30182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:49.122056 30182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:49.123157 30182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "efcedff3810b4268a236182a7e50b0e4"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "7a7d96f86f339792c7df426d4839f564"
server_key_iv: "adc25a41a1c230350e2ae8738aa82e95"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.123602 30182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "efcedff3810b4268a236182a7e50b0e4"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "7a7d96f86f339792c7df426d4839f564"
server_key_iv: "adc25a41a1c230350e2ae8738aa82e95"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.125064 30182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:49.125803 30197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.125973 30182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:49.126076 30182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "efcedff3810b4268a236182a7e50b0e4"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "7a7d96f86f339792c7df426d4839f564"
server_key_iv: "adc25a41a1c230350e2ae8738aa82e95"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.126152 30182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:49.138084 30182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:49.140512 30182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:49.140686 30182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:49.141218 30182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:49.141283 30182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.141353 30182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:49.141389 30182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:49.148794 30182 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:41465
I20260925 14:45:49.149236 30182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:49.149232 30313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:41465 every 8 connection(s)
I20260925 14:45:49.151454 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.141776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59685 (local address 127.21.205.254:33615)
0925 14:45:49.141820 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.141823 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.142102 (+ 279us) server_negotiation.cc:407] Connection header received
0925 14:45:49.142622 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.142623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.142639 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.142715 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.143644 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.144301 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.145373 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.145585 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.145672 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.148586 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.148588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.148588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.148598 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.150273 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.150655 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.150656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.150656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.150710 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.150909 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.150909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.150910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.151019 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.151067 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.151307 (+ 240us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.151390 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:45:49.151612 30206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.141773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.194:59685)
0925 14:45:49.141965 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.142003 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.142513 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.142828 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.142832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.143199 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.143546 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.143554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.144413 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.144414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.145277 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.145280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.145685 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.145686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.145838 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.146463 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.146475 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.148475 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.150351 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.150355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.150357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.150585 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.150764 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.150765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.150765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.150859 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.151070 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.151074 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.151197 (+ 123us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.151281 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.151411 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260925 14:45:49.152113 30314 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33615
I20260925 14:45:49.152199 30314 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:49.152410 30314 heartbeater.cc:511] Master 127.21.205.254:33615 requested a full tablet report, sending...
I20260925 14:45:49.152819 29968 ts_manager.cc:194] Registered new tserver with Master: efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465)
I20260925 14:45:49.153292 29968 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:59685
I20260925 14:45:49.158388 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30182
I20260925 14:45:49.158507 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:45:49.158687 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.153386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:35553 (local address 127.21.205.254:33615)
0925 14:45:49.153440 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.153442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.153451 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:49.153471 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.153472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.153482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.153533 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.153889 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.154420 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.155095 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.155267 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.155314 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.156235 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.156236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.156236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.156242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.157604 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.157893 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.157894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.157894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.157941 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.158090 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.158091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.158091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.158303 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.158383 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.158512 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.158591 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 14:45:49.158699 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.153121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.194:35553)
0925 14:45:49.153245 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.153255 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.153305 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.153545 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.153546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.153743 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.153804 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.153806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.154514 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.154515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.155030 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.155032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.155324 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.155325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.155397 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.155860 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.155865 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.156161 (+ 296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.157673 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.157674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.157675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.157835 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.157995 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.157995 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.157996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.158044 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.158372 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.158373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.158410 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.158483 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.158595 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260925 14:45:49.158722 22327 external_mini_cluster.cc:1428] Setting key 5057bcd24519bdb8edf568476213df4e
WARNING: no policy specified for oryx/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
I20260925 14:45:49.182958 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33615
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:49.267396 30323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:49.267700 30323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:49.267732 30323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:49.269025 30323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:49.269080 30323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:49.269104 30323 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 14:45:49.269161 30323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:49.270830 30323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.205.254:33615
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:49.271106 30323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:49.271368 30323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:49.276662 30331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:49.276662 30328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:49.276646 30329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:49.276693 30323 server_base.cc:1034] running on GCE node
I20260925 14:45:49.277140 30323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:49.277384 30323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:49.278528 30323 hybrid_clock.cc:648] HybridClock initialized: now 1790347549278510 us; error 37 us; skew 500 ppm
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:49.281239 30323 init.cc:377] Logged in from keytab as oryx/127.21.205.195@KRBTEST.COM (short username oryx)
I20260925 14:45:49.281697 30323 webserver.cc:460] Webserver started at http://127.21.205.195:44603/ using document root <none> and password file <none>
I20260925 14:45:49.281953 30323 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:49.282001 30323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:49.282171 30323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:49.283269 30323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "0e6652b414014e6dab567179545ef7d9"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "6a1d658b0e177e8f3c7120952fada2fb"
server_key_iv: "1b4ab3cd283cd7b61805b7b183e2a0fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.283677 30323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "0e6652b414014e6dab567179545ef7d9"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "6a1d658b0e177e8f3c7120952fada2fb"
server_key_iv: "1b4ab3cd283cd7b61805b7b183e2a0fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.285326 30323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:49.286185 30338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.286357 30323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.286451 30323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "0e6652b414014e6dab567179545ef7d9"
format_stamp: "Formatted at 2026-09-25 14:45:49 on dist-test-slave-d7lw"
server_key: "6a1d658b0e177e8f3c7120952fada2fb"
server_key_iv: "1b4ab3cd283cd7b61805b7b183e2a0fe"
server_key_version: "encryptionkey@0"
I20260925 14:45:49.286510 30323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:49.311930 30323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:49.314350 30323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:49.314479 30323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:49.314983 30323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:49.315021 30323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.315049 30323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:49.315104 30323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.321851 30323 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:40137
I20260925 14:45:49.322039 30453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:40137 every 8 connection(s)
I20260925 14:45:49.322309 30323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:49.325785 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.315642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:36995 (local address 127.21.205.254:33615)
0925 14:45:49.315703 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.315705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.315964 (+ 259us) server_negotiation.cc:407] Connection header received
0925 14:45:49.316573 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.316575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.316588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.316649 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.317583 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.318460 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.320151 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.320506 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.320600 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.323304 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.323306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.323306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.323313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.324604 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.324996 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.324997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.324997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.325048 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.325252 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.325252 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.325253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.325362 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.325413 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.325651 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.325719 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:45:49.325987 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.315602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.195:36995)
0925 14:45:49.315820 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.315870 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.316462 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.316732 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.316736 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.317130 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.317457 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.317464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.318573 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.318575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.320053 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.320056 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.320608 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.320610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.320764 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.321351 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.321362 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.323204 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.324678 (+ 1474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.324685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.324687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.324931 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.325103 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.325104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.325104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.325202 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.325413 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.325417 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.325539 (+ 122us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.325626 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.325754 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260925 14:45:49.326467 30454 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33615
I20260925 14:45:49.326555 30454 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:49.326771 30454 heartbeater.cc:511] Master 127.21.205.254:33615 requested a full tablet report, sending...
I20260925 14:45:49.327312 29967 ts_manager.cc:194] Registered new tserver with Master: 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:49.327827 29967 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:36995
I20260925 14:45:49.328589 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30323
I20260925 14:45:49.328682 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:49.328914 22327 external_mini_cluster.cc:1428] Setting key 40374fa1243d54a5165b0abf058788d1
I20260925 14:45:49.329895 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:45:49.333848 30460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.327542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.21.205.195:43005)
0925 14:45:49.327700 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.327708 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.327757 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.327950 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.327951 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.328148 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.328222 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.328224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.328945 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.328946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.329676 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.329679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.330139 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.330140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.330209 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.330668 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.330674 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.331105 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.332769 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.332770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.332771 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.333020 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.333216 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.333217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.333217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.333278 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.333591 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.333592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.333634 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.333685 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.333773 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":65,"threads_started":1}
I20260925 14:45:49.333952 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.327595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:43005 (local address 127.21.205.254:33615)
0925 14:45:49.327641 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.327643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.327762 (+ 119us) server_negotiation.cc:407] Connection header received
0925 14:45:49.327830 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.327831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.327843 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.327914 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.328299 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.328865 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.329746 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.330028 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.330138 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.331193 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.331195 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.331195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.331201 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.332662 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.333083 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.333085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.333085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.333146 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.333364 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.333365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.333365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.333527 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.333597 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.333728 (+ 131us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.333814 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 14:45:49.352847 30018 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:49.347370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36590 (local address 127.21.205.254:33615)
0925 14:45:49.347436 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.347438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.347448 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:45:49.347475 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.347476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.347485 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.347535 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.348175 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.348656 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.349357 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.349575 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.349623 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.352736 (+ 3113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36590: BlockingRecv error: recv got EOF from 127.0.0.1:36590 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347549, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:49.360376 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.353723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.21.205.254:33615)
0925 14:45:49.353786 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.353788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.353843 (+ 55us) server_negotiation.cc:407] Connection header received
0925 14:45:49.353924 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.353925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.353933 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.353976 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.354432 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.354899 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.355554 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.355727 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.355815 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.357934 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.357935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.357936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.357944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.359344 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.359675 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.359676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.359677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.359725 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.359928 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.359929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.359929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.360039 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.360117 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.360252 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.360310 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:45:49.362107 29967 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:49.363749 29967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:49.370682 30477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.0.0.1:32768)
0925 14:45:49.366488 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.366532 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.366607 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.366940 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.366941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.366943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.367008 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.367011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.368018 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.368020 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.369533 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.369536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370483 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.370541 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370545 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.370579 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.370612 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":485,"thread_start_us":54,"threads_started":1}
I20260925 14:45:49.370766 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32768 (local address 127.21.205.193:37669)
0925 14:45:49.366584 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.366589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.366601 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:49.366708 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.366710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.366728 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.366812 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.367096 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.367936 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.369633 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370406 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370511 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370569 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.370609 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.370648 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":81,"threads_started":1}
I20260925 14:45:49.370884 30476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:41465 (local address 127.0.0.1:58434)
0925 14:45:49.366821 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.366831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.366915 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.367188 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.367189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.367191 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.367238 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.367240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.368486 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.368487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.369552 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.369554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370608 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.370686 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370690 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.370741 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.370791 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260925 14:45:49.370682 30480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40137 (local address 127.0.0.1:50844)
0925 14:45:49.366561 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.366571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.366627 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.366914 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.366917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.366920 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.367002 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.367004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.368346 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.368347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.369550 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.369554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370318 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.370413 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370428 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.370501 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.370565 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":4,"thread_start_us":85,"threads_started":1}
I20260925 14:45:49.371173 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58434 (local address 127.21.205.194:41465)
0925 14:45:49.366756 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.366766 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.366888 (+ 122us) server_negotiation.cc:407] Connection header received
0925 14:45:49.367023 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.367025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.367042 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.367097 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.367324 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.368419 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.369727 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370432 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370515 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370569 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.371012 (+ 443us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.371069 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260925 14:45:49.371636 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.366282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.21.205.195:40137)
0925 14:45:49.366487 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.366493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.366702 (+ 209us) server_negotiation.cc:407] Connection header received
0925 14:45:49.366742 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.366744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.366761 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.366843 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.367088 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.368262 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.369664 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.370226 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.370340 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.370406 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.370722 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.371554 (+ 832us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260925 14:45:49.371963 30247 tablet_service.cc:1467] Processing CreateTablet for tablet ddb63db814134c86a07c2d1db740f6a7 (DEFAULT_TABLE table=test-table [id=4230754621464bb1bdf490fb53abbd1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.372008 30106 tablet_service.cc:1467] Processing CreateTablet for tablet ddb63db814134c86a07c2d1db740f6a7 (DEFAULT_TABLE table=test-table [id=4230754621464bb1bdf490fb53abbd1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.372344 30247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb63db814134c86a07c2d1db740f6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.372375 30106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb63db814134c86a07c2d1db740f6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.375686 30483 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Bootstrap starting.
I20260925 14:45:49.375685 30482 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Bootstrap starting.
I20260925 14:45:49.376363 30482 tablet_bootstrap.cc:654] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.376435 30483 tablet_bootstrap.cc:654] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.376281 30388 tablet_service.cc:1467] Processing CreateTablet for tablet ddb63db814134c86a07c2d1db740f6a7 (DEFAULT_TABLE table=test-table [id=4230754621464bb1bdf490fb53abbd1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.376681 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ddb63db814134c86a07c2d1db740f6a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.376699 30482 log.cc:824] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:49.376763 30483 log.cc:824] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:49.377566 30482 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: No bootstrap required, opened a new log
I20260925 14:45:49.377671 30482 ts_tablet_manager.cc:1397] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:49.379850 30482 raft_consensus.cc:348] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.380046 30482 raft_consensus.cc:374] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.380084 30482 raft_consensus.cc:729] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.380215 30482 consensus_queue.cc:260] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.380250 30483 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: No bootstrap required, opened a new log
I20260925 14:45:49.380338 30483 ts_tablet_manager.cc:1397] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 14:45:49.380499 30482 ts_tablet_manager.cc:1428] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:49.380645 30314 heartbeater.cc:503] Master 127.21.205.254:33615 was elected leader, sending a full tablet report...
I20260925 14:45:49.380528 30486 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Bootstrap starting.
I20260925 14:45:49.381970 30486 tablet_bootstrap.cc:654] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.382310 30486 log.cc:824] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:49.382695 30483 raft_consensus.cc:348] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.382901 30483 raft_consensus.cc:374] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.382944 30483 raft_consensus.cc:729] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.383073 30483 consensus_queue.cc:260] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.383173 30486 tablet_bootstrap.cc:492] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: No bootstrap required, opened a new log
I20260925 14:45:49.383239 30486 ts_tablet_manager.cc:1397] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 14:45:49.383379 30483 ts_tablet_manager.cc:1428] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:49.383412 30173 heartbeater.cc:503] Master 127.21.205.254:33615 was elected leader, sending a full tablet report...
I20260925 14:45:49.385406 30486 raft_consensus.cc:348] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.385555 30486 raft_consensus.cc:374] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.385576 30486 raft_consensus.cc:729] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.385720 30486 consensus_queue.cc:260] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.386018 30486 ts_tablet_manager.cc:1428] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:45:49.386039 30454 heartbeater.cc:503] Master 127.21.205.254:33615 was elected leader, sending a full tablet report...
W20260925 14:45:49.399997 30315 tablet.cc:2369] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:49.413390 30491 raft_consensus.cc:482] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:49.413519 30491 raft_consensus.cc:504] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.413864 30491 leader_election.cc:288] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465)
I20260925 14:45:49.416802 30478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.414126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:57893 (local address 127.21.205.193:37669)
0925 14:45:49.414176 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.414178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.414205 (+ 27us) server_negotiation.cc:407] Connection header received
0925 14:45:49.414236 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.414237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.414246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.414317 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.414478 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.414961 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416052 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.416428 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416513 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.416524 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.416684 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.416725 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:45:49.416831 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.414147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:53049 (local address 127.21.205.194:41465)
0925 14:45:49.414192 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.414194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.414326 (+ 132us) server_negotiation.cc:407] Connection header received
0925 14:45:49.414403 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.414404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.414413 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.414480 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.414631 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.415095 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416086 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.416466 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416547 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.416555 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.416713 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.416750 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:45:49.416846 30492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.414049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:41465 (local address 127.21.205.195:53049)
0925 14:45:49.414261 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.414268 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.414339 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.414484 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.414485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.414486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.414562 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.414564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.415178 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.415179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.415989 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.415991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.416558 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.416638 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.416641 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.416705 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.416746 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":130,"threads_started":1}
I20260925 14:45:49.416848 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.414033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.21.205.195:57893)
0925 14:45:49.414094 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.414107 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.414151 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.414325 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.414326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.414328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.414406 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.414407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.415049 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.415050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.415957 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.415960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.416512 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.416513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.416604 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.416607 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.416665 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.416708 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":28}
I20260925 14:45:49.417243 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb63db814134c86a07c2d1db740f6a7" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4" is_pre_election: true
I20260925 14:45:49.417449 30267 raft_consensus.cc:2454] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 0.
I20260925 14:45:49.417553 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb63db814134c86a07c2d1db740f6a7" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96" is_pre_election: true
I20260925 14:45:49.417716 30126 raft_consensus.cc:2454] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 0.
I20260925 14:45:49.417725 30340 leader_election.cc:302] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6652b414014e6dab567179545ef7d9, efcedff3810b4268a236182a7e50b0e4; no voters:
I20260925 14:45:49.417883 30491 raft_consensus.cc:2781] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:49.417943 30491 raft_consensus.cc:482] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:49.417966 30491 raft_consensus.cc:3037] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.419008 30491 raft_consensus.cc:504] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.419183 30491 leader_election.cc:288] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 election: Requested vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465)
I20260925 14:45:49.419384 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb63db814134c86a07c2d1db740f6a7" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96"
I20260925 14:45:49.419384 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ddb63db814134c86a07c2d1db740f6a7" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4"
I20260925 14:45:49.419504 30126 raft_consensus.cc:3037] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.419504 30267 raft_consensus.cc:3037] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.420486 30267 raft_consensus.cc:2454] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 1.
I20260925 14:45:49.420486 30126 raft_consensus.cc:2454] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 1.
I20260925 14:45:49.420717 30340 leader_election.cc:302] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6652b414014e6dab567179545ef7d9, efcedff3810b4268a236182a7e50b0e4; no voters:
I20260925 14:45:49.420872 30491 raft_consensus.cc:2781] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:49.421080 30491 raft_consensus.cc:686] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Running, Role: LEADER
I20260925 14:45:49.421255 30491 consensus_queue.cc:237] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } }
I20260925 14:45:49.422336 29967 catalog_manager.cc:5495] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e6652b414014e6dab567179545ef7d9 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "0e6652b414014e6dab567179545ef7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } health_report { overall_health: UNKNOWN } } }
I20260925 14:45:49.429039 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.426439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.21.205.195:40137)
0925 14:45:49.426495 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.426497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.426504 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:49.426540 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.426541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.426566 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.426631 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:49.426765 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.427250 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.428123 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.428393 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.428463 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.428599 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:49.428710 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:49.428799 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.428870 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.428915 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:45:49.431186 30126 raft_consensus.cc:1261] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Refusing update from remote peer 0e6652b414014e6dab567179545ef7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:49.431186 30267 raft_consensus.cc:1261] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Refusing update from remote peer 0e6652b414014e6dab567179545ef7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:49.431566 30493 consensus_queue.cc:1035] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:49.431689 30491 consensus_queue.cc:1035] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:49.434579 30496 mvcc.cc:205] Tried to move back new op lower bound from 7333263562466783232 to 7333263562430398464. Current Snapshot: MvccSnapshot[applied={T|T < 7333263562466783232}]
I20260925 14:45:49.435276 30497 mvcc.cc:205] Tried to move back new op lower bound from 7333263562466783232 to 7333263562430398464. Current Snapshot: MvccSnapshot[applied={T|T < 7333263562466783232}]
I20260925 14:45:49.435930 30498 mvcc.cc:205] Tried to move back new op lower bound from 7333263562466783232 to 7333263562430398464. Current Snapshot: MvccSnapshot[applied={T|T < 7333263562466783232 or (T in {7333263562466783232})}]
I20260925 14:45:49.437589 29968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:49.437711 29968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:49.438611 29968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Sending DeleteTablet for 3 replicas of tablet ddb63db814134c86a07c2d1db740f6a7
I20260925 14:45:49.439046 30388 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb63db814134c86a07c2d1db740f6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:49 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:50844
I20260925 14:45:49.439205 30247 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb63db814134c86a07c2d1db740f6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:49 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260925 14:45:49.439222 29968 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:49.439390 29968 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:49.439462 30507 tablet_replica.cc:330] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: stopping tablet replica
I20260925 14:45:49.439529 30106 tablet_service.cc:1514] Processing DeleteTablet for tablet ddb63db814134c86a07c2d1db740f6a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:49 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32768
I20260925 14:45:49.439576 30507 raft_consensus.cc:2229] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:49.439714 30507 raft_consensus.cc:2258] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:49.439957 30506 tablet_replica.cc:330] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: stopping tablet replica
I20260925 14:45:49.440069 30506 raft_consensus.cc:2229] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:49.440207 30506 raft_consensus.cc:2258] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:49.440402 30508 tablet_replica.cc:330] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: stopping tablet replica
I20260925 14:45:49.440537 30508 raft_consensus.cc:2229] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:49.440680 30508 raft_consensus.cc:2258] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:49.441159 30506 ts_tablet_manager.cc:1905] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:49.441350 30247 tablet_service.cc:1467] Processing CreateTablet for tablet 82bff053f285487e94c5a9a5c5dab55d (DEFAULT_TABLE table=test-table [id=3754204655ce4ba9894d274d0f1d2f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.441479 30247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82bff053f285487e94c5a9a5c5dab55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.441672 30106 tablet_service.cc:1467] Processing CreateTablet for tablet 82bff053f285487e94c5a9a5c5dab55d (DEFAULT_TABLE table=test-table [id=3754204655ce4ba9894d274d0f1d2f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.441802 30106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82bff053f285487e94c5a9a5c5dab55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.443046 30482 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Bootstrap starting.
I20260925 14:45:49.443040 30506 ts_tablet_manager.cc:1918] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:49.443128 30506 log.cc:1196] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/ddb63db814134c86a07c2d1db740f6a7
I20260925 14:45:49.443231 30388 tablet_service.cc:1467] Processing CreateTablet for tablet 82bff053f285487e94c5a9a5c5dab55d (DEFAULT_TABLE table=test-table [id=3754204655ce4ba9894d274d0f1d2f5c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:49.443337 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82bff053f285487e94c5a9a5c5dab55d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.443459 30506 ts_tablet_manager.cc:1939] T ddb63db814134c86a07c2d1db740f6a7 P 0e6652b414014e6dab567179545ef7d9: Deleting consensus metadata
I20260925 14:45:49.443549 30482 tablet_bootstrap.cc:654] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.444324 30508 ts_tablet_manager.cc:1905] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:49.444581 30483 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Bootstrap starting.
I20260925 14:45:49.445015 30483 tablet_bootstrap.cc:654] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.445399 30486 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Bootstrap starting.
I20260925 14:45:49.445844 30486 tablet_bootstrap.cc:654] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.445835 30483 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: No bootstrap required, opened a new log
I20260925 14:45:49.445899 30483 ts_tablet_manager.cc:1397] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.446014 30508 ts_tablet_manager.cc:1918] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:49.446079 30508 log.cc:1196] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/ddb63db814134c86a07c2d1db740f6a7
I20260925 14:45:49.446076 30483 raft_consensus.cc:348] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.446188 30483 raft_consensus.cc:374] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.446219 30483 raft_consensus.cc:729] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.446264 30483 consensus_queue.cc:260] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.446319 29953 catalog_manager.cc:4841] TS 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137): tablet ddb63db814134c86a07c2d1db740f6a7 (table test-table [id=4230754621464bb1bdf490fb53abbd1c]) successfully deleted
I20260925 14:45:49.446377 30483 ts_tablet_manager.cc:1428] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.446434 30508 ts_tablet_manager.cc:1939] T ddb63db814134c86a07c2d1db740f6a7 P f3bed538261d4528a6d843c0855d2b96: Deleting consensus metadata
I20260925 14:45:49.446995 29955 catalog_manager.cc:4841] TS f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669): tablet ddb63db814134c86a07c2d1db740f6a7 (table test-table [id=4230754621464bb1bdf490fb53abbd1c]) successfully deleted
I20260925 14:45:49.447335 30486 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: No bootstrap required, opened a new log
I20260925 14:45:49.447396 30486 ts_tablet_manager.cc:1397] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:49.447577 30486 raft_consensus.cc:348] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.447672 30486 raft_consensus.cc:374] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.447703 30486 raft_consensus.cc:729] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.447755 30486 consensus_queue.cc:260] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.447916 30486 ts_tablet_manager.cc:1428] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:49.448347 30507 ts_tablet_manager.cc:1905] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:49.450033 30507 ts_tablet_manager.cc:1918] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:49.450132 30482 tablet_bootstrap.cc:492] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: No bootstrap required, opened a new log
I20260925 14:45:49.450132 30507 log.cc:1196] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/ddb63db814134c86a07c2d1db740f6a7
I20260925 14:45:49.450218 30482 ts_tablet_manager.cc:1397] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260925 14:45:49.450421 30482 raft_consensus.cc:348] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.450541 30482 raft_consensus.cc:374] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.450546 30507 ts_tablet_manager.cc:1939] T ddb63db814134c86a07c2d1db740f6a7 P efcedff3810b4268a236182a7e50b0e4: Deleting consensus metadata
I20260925 14:45:49.450585 30482 raft_consensus.cc:729] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.450646 30482 consensus_queue.cc:260] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.450770 30482 ts_tablet_manager.cc:1428] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.451416 29953 catalog_manager.cc:4841] TS efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465): tablet ddb63db814134c86a07c2d1db740f6a7 (table test-table [id=4230754621464bb1bdf490fb53abbd1c]) successfully deleted
W20260925 14:45:49.495253 30174 tablet.cc:2369] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:45:49.545595 30490 raft_consensus.cc:482] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:49.545709 30490 raft_consensus.cc:504] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.546033 30490 leader_election.cc:288] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:49.549753 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.546358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:35927 (local address 127.21.205.194:41465)
0925 14:45:49.546420 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.546422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.546540 (+ 118us) server_negotiation.cc:407] Connection header received
0925 14:45:49.546661 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.546662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.546671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.546704 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.546929 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.547692 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.548730 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.549332 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.549419 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.549431 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.549610 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.549657 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:45:49.549752 30513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.546272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:41465 (local address 127.21.205.193:35927)
0925 14:45:49.546453 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.546465 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.546574 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.546788 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.546788 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.546791 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.546852 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.546854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.547826 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.547827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.548650 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.548655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.549420 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.549421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.549497 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.549503 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.549579 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.549633 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260925 14:45:49.550289 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bff053f285487e94c5a9a5c5dab55d" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4" is_pre_election: true
I20260925 14:45:49.550437 30267 raft_consensus.cc:2454] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:49.550635 30058 leader_election.cc:302] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:49.550735 30514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.546476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40137 (local address 127.21.205.193:46093)
0925 14:45:49.546544 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.546554 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.546599 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.546748 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.546749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.546752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:49.546842 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.546845 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.548660 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.548661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.549525 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.549540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.550352 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.550353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.550440 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.550445 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.550601 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.550645 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"thread_start_us":41,"threads_started":1}
I20260925 14:45:49.550741 30479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.546575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:46093 (local address 127.21.205.195:40137)
0925 14:45:49.546624 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.546625 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.546635 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:45:49.546663 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.546664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.546674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.546734 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:49.547847 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.548568 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.549632 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.550280 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.550350 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.550359 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.550628 (+ 269us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.550676 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:45:49.550777 30490 raft_consensus.cc:2781] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:49.550809 30490 raft_consensus.cc:482] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:49.550830 30490 raft_consensus.cc:3037] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.551647 30490 raft_consensus.cc:504] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.551698 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bff053f285487e94c5a9a5c5dab55d" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9" is_pre_election: true
I20260925 14:45:49.551836 30490 leader_election.cc:288] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Requested vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:49.551851 30396 raft_consensus.cc:2454] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:49.552084 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bff053f285487e94c5a9a5c5dab55d" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4"
I20260925 14:45:49.552131 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82bff053f285487e94c5a9a5c5dab55d" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9"
I20260925 14:45:49.552191 30396 raft_consensus.cc:3037] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.552170 30267 raft_consensus.cc:3037] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.553015 30396 raft_consensus.cc:2454] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 1.
I20260925 14:45:49.553015 30267 raft_consensus.cc:2454] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 1.
I20260925 14:45:49.553185 30058 leader_election.cc:302] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:49.553313 30490 raft_consensus.cc:2781] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:49.553439 30490 raft_consensus.cc:686] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 1 LEADER]: Becoming Leader. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Running, Role: LEADER
I20260925 14:45:49.553509 30490 consensus_queue.cc:237] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.554236 29967 catalog_manager.cc:5495] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3bed538261d4528a6d843c0855d2b96 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } health_report { overall_health: UNKNOWN } } }
W20260925 14:45:49.572962 30455 tablet.cc:2369] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 14:45:49 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347548, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.254@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:49.591594 30018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.584771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36612 (local address 127.21.205.254:33615)
0925 14:45:49.584845 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:49.584847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:49.584941 (+ 94us) server_negotiation.cc:407] Connection header received
0925 14:45:49.584993 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:49.584993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:49.585003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:49.585054 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:49.585642 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.586148 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.586843 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.587037 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.587093 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.589184 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:49.589186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:49.589186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:49.589193 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:49.590515 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.590870 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.590871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.590871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.590919 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:49.591075 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:49.591075 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:49.591075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:49.591179 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:49.591230 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:49.591439 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 14:45:49.591525 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 14:45:49.591715 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:49.584713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33615 (local address 127.0.0.1:36612)
0925 14:45:49.584861 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:49.584878 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:49.584947 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:49.585103 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:49.585104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:49.585503 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:49.585556 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.585559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.586218 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.586219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.586756 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:49.586758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:49.587106 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:49.587107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:49.587150 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:49.587659 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:49.587664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:49.589094 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:49.590586 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.590594 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.590596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.590797 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.590972 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:49.590972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:49.590972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:49.591027 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:49.591259 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:49.591268 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:49.591321 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 14:45:49.591399 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 14:45:49.591608 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":68,"thread_start_us":96,"threads_started":1}
I20260925 14:45:49.593824 29967 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:36612:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 14:45:49.594059 29967 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:49.595492 30106 tablet_service.cc:1467] Processing CreateTablet for tablet 24a856b6383e4915b3893af9c31c8030 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddf636e4e144447fa3fbcea6ba4c52dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:45:49.595508 30247 tablet_service.cc:1467] Processing CreateTablet for tablet 24a856b6383e4915b3893af9c31c8030 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddf636e4e144447fa3fbcea6ba4c52dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:45:49.595547 30388 tablet_service.cc:1467] Processing CreateTablet for tablet 24a856b6383e4915b3893af9c31c8030 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ddf636e4e144447fa3fbcea6ba4c52dc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 14:45:49.595641 30106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a856b6383e4915b3893af9c31c8030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.595659 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a856b6383e4915b3893af9c31c8030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.595659 30247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24a856b6383e4915b3893af9c31c8030. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:49.597004 30483 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: Bootstrap starting.
I20260925 14:45:49.597169 30482 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: Bootstrap starting.
I20260925 14:45:49.597169 30486 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: Bootstrap starting.
I20260925 14:45:49.597421 30483 tablet_bootstrap.cc:654] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.597496 30486 tablet_bootstrap.cc:654] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.597595 30482 tablet_bootstrap.cc:654] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:49.598152 30486 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: No bootstrap required, opened a new log
I20260925 14:45:49.598210 30486 ts_tablet_manager.cc:1397] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.598225 30483 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: No bootstrap required, opened a new log
I20260925 14:45:49.598264 30483 ts_tablet_manager.cc:1397] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.598311 30482 tablet_bootstrap.cc:492] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: No bootstrap required, opened a new log
I20260925 14:45:49.598366 30482 ts_tablet_manager.cc:1397] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:49.598356 30486 raft_consensus.cc:348] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598440 30486 raft_consensus.cc:374] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.598404 30483 raft_consensus.cc:348] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598467 30486 raft_consensus.cc:729] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.598474 30483 raft_consensus.cc:374] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.598495 30483 raft_consensus.cc:729] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.598515 30486 consensus_queue.cc:260] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598533 30483 consensus_queue.cc:260] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598526 30482 raft_consensus.cc:348] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598604 30482 raft_consensus.cc:374] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:49.598608 30486 ts_tablet_manager.cc:1428] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:49.598620 30483 ts_tablet_manager.cc:1428] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:49.598623 30482 raft_consensus.cc:729] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Initialized, Role: FOLLOWER
I20260925 14:45:49.598671 30482 consensus_queue.cc:260] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.598762 30482 ts_tablet_manager.cc:1428] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:49.598886 30491 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:49.598866 30488 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:49.598985 30491 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:49.598991 30488 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:49.599117 30490 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:49.599198 30490 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:49.969934 30490 raft_consensus.cc:482] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:49.970077 30490 raft_consensus.cc:504] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.970257 30490 leader_election.cc:288] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:49.970657 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a856b6383e4915b3893af9c31c8030" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4" is_pre_election: true
I20260925 14:45:49.970657 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a856b6383e4915b3893af9c31c8030" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9" is_pre_election: true
I20260925 14:45:49.970770 30267 raft_consensus.cc:2454] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:49.970770 30396 raft_consensus.cc:2454] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:49.970923 30058 leader_election.cc:302] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:49.971015 30490 raft_consensus.cc:2781] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:49.971057 30490 raft_consensus.cc:482] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:49.971083 30490 raft_consensus.cc:3037] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.971742 30490 raft_consensus.cc:504] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.971899 30490 leader_election.cc:288] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Requested vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:49.972098 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a856b6383e4915b3893af9c31c8030" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4"
I20260925 14:45:49.972194 30267 raft_consensus.cc:3037] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.972183 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24a856b6383e4915b3893af9c31c8030" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9"
I20260925 14:45:49.972285 30396 raft_consensus.cc:3037] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:49.972837 30267 raft_consensus.cc:2454] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 1.
I20260925 14:45:49.972990 30396 raft_consensus.cc:2454] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 1.
I20260925 14:45:49.973094 30058 leader_election.cc:302] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:49.973201 30490 raft_consensus.cc:2781] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:49.973278 30490 raft_consensus.cc:686] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [term 1 LEADER]: Becoming Leader. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Running, Role: LEADER
I20260925 14:45:49.973345 30490 consensus_queue.cc:237] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:49.973642 30515 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: TxnStatusTablet state changed. Reason: New leader f3bed538261d4528a6d843c0855d2b96. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:49.973727 30515 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:45:49.973879 30527 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 14:45:49.974000 30527 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 14:45:49.974044 29966 catalog_manager.cc:5495] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3bed538261d4528a6d843c0855d2b96 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } health_report { overall_health: UNKNOWN } } }
I20260925 14:45:50.022545 30515 consensus_queue.cc:1035] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.024554 30515 consensus_queue.cc:1035] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.042563 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.039239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.0.0.1:32772)
0925 14:45:50.039355 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.039373 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.039437 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.039901 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.039903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.039904 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:45:50.039970 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.039973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.040688 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.040689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.041291 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.041292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.041673 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.041674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.041749 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.041765 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.042334 (+ 569us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.042347 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.042413 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.042474 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 14:45:50.042572 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.039522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.21.205.193:37669)
0925 14:45:50.039692 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.039695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.039707 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:50.039751 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.039752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.039771 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.039890 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.040054 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.040599 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.041380 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.041594 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.041673 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.042153 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.042257 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.042347 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.042418 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.042444 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":122,"threads_started":1}
I20260925 14:45:50.043753 30396 raft_consensus.cc:1261] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Refusing update from remote peer f3bed538261d4528a6d843c0855d2b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.043746 30267 raft_consensus.cc:1261] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Refusing update from remote peer f3bed538261d4528a6d843c0855d2b96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.044060 30490 consensus_queue.cc:1035] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.044207 30515 consensus_queue.cc:1035] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.044714 30491 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: TxnStatusTablet state changed. Reason: New leader f3bed538261d4528a6d843c0855d2b96. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.044821 30491 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:50.044874 30488 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: TxnStatusTablet state changed. Reason: New leader f3bed538261d4528a6d843c0855d2b96. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.044986 30488 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:50.045636 30490 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.045727 30490 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:45:50.045747 30488 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.045828 30488 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P efcedff3810b4268a236182a7e50b0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:50.046948 30515 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.047053 30515 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P f3bed538261d4528a6d843c0855d2b96: This TxnStatusTablet replica's current role is: LEADER
I20260925 14:45:50.048156 30491 tablet_replica.cc:439] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f3bed538261d4528a6d843c0855d2b96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } } }
I20260925 14:45:50.048272 30491 tablet_replica.cc:442] T 24a856b6383e4915b3893af9c31c8030 P 0e6652b414014e6dab567179545ef7d9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 14:45:50.050464 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.047602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.21.205.193:37669)
0925 14:45:50.047644 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.047646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.047720 (+ 74us) server_negotiation.cc:407] Connection header received
0925 14:45:50.047833 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.047834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.047845 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.047874 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.048103 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.048845 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.049632 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.049950 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.050026 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.050196 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.050232 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.050290 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.050341 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.050374 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:45:50.055805 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.053095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:38457 (local address 127.21.205.193:37669)
0925 14:45:50.053137 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.053139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.053253 (+ 114us) server_negotiation.cc:407] Connection header received
0925 14:45:50.053298 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.053299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.053309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.053398 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.053520 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.054249 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.054992 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.055273 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.055353 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.055508 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.055543 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.055601 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.055647 (+ 46us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.055669 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:45:50.055836 30546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.053034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.21.205.193:38457)
0925 14:45:50.053180 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.053191 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.053247 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.053432 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.053432 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.053434 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:45:50.053475 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.053477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.054327 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.054328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.054907 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.054910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.055364 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.055365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.055417 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.055431 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.055606 (+ 175us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.055608 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.055641 (+ 33us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.055678 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260925 14:45:50.072582 29966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.072721 29966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.073511 29966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Sending DeleteTablet for 3 replicas of tablet 82bff053f285487e94c5a9a5c5dab55d
I20260925 14:45:50.073845 30105 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bff053f285487e94c5a9a5c5dab55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32768
I20260925 14:45:50.074077 30247 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bff053f285487e94c5a9a5c5dab55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260925 14:45:50.074311 30388 tablet_service.cc:1514] Processing DeleteTablet for tablet 82bff053f285487e94c5a9a5c5dab55d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:50844
I20260925 14:45:50.074352 30549 tablet_replica.cc:330] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: stopping tablet replica
I20260925 14:45:50.074487 30550 tablet_replica.cc:330] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: stopping tablet replica
I20260925 14:45:50.074556 30550 raft_consensus.cc:2229] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.074517 30549 raft_consensus.cc:2229] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:50.074610 30550 raft_consensus.cc:2258] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.074766 30549 raft_consensus.cc:2258] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.074918 30550 ts_tablet_manager.cc:1905] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.075142 30549 ts_tablet_manager.cc:1905] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.076906 30550 ts_tablet_manager.cc:1918] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:50.076974 30550 log.cc:1196] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/82bff053f285487e94c5a9a5c5dab55d
I20260925 14:45:50.077201 30550 ts_tablet_manager.cc:1939] T 82bff053f285487e94c5a9a5c5dab55d P efcedff3810b4268a236182a7e50b0e4: Deleting consensus metadata
I20260925 14:45:50.077441 30549 ts_tablet_manager.cc:1918] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:50.077558 29953 catalog_manager.cc:4841] TS efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465): tablet 82bff053f285487e94c5a9a5c5dab55d (table test-table [id=3754204655ce4ba9894d274d0f1d2f5c]) successfully deleted
I20260925 14:45:50.077582 30549 log.cc:1196] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/82bff053f285487e94c5a9a5c5dab55d
I20260925 14:45:50.077952 30549 ts_tablet_manager.cc:1939] T 82bff053f285487e94c5a9a5c5dab55d P f3bed538261d4528a6d843c0855d2b96: Deleting consensus metadata
I20260925 14:45:50.078366 29955 catalog_manager.cc:4841] TS f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669): tablet 82bff053f285487e94c5a9a5c5dab55d (table test-table [id=3754204655ce4ba9894d274d0f1d2f5c]) successfully deleted
I20260925 14:45:50.080992 30551 tablet_replica.cc:330] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: stopping tablet replica
I20260925 14:45:50.081091 30551 raft_consensus.cc:2229] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.081147 30551 raft_consensus.cc:2258] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.082638 30551 ts_tablet_manager.cc:1905] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.084959 30551 ts_tablet_manager.cc:1918] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:50.085041 30551 log.cc:1196] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/82bff053f285487e94c5a9a5c5dab55d
I20260925 14:45:50.085351 30551 ts_tablet_manager.cc:1939] T 82bff053f285487e94c5a9a5c5dab55d P 0e6652b414014e6dab567179545ef7d9: Deleting consensus metadata
I20260925 14:45:50.085798 29953 catalog_manager.cc:4841] TS 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137): tablet 82bff053f285487e94c5a9a5c5dab55d (table test-table [id=3754204655ce4ba9894d274d0f1d2f5c]) successfully deleted
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[29923](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 14:45:50.100301 30559 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:50.093971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.21.205.254:33615)
0925 14:45:50.094106 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.094108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.094144 (+ 36us) server_negotiation.cc:407] Connection header received
0925 14:45:50.094220 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.094221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.094235 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.094325 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:50.094921 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.095722 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.096542 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.096848 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.096930 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.100221 (+ 3291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36618: BlockingRecv error: recv got EOF from 127.0.0.1:36618 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":75,"threads_started":1}
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[29923](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM
I20260925 14:45:50.109233 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.101162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.21.205.254:33615)
0925 14:45:50.101206 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.101208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.101276 (+ 68us) server_negotiation.cc:407] Connection header received
0925 14:45:50.101364 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.101365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.101380 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.101445 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:50.101893 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.102700 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.103405 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.103704 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.103808 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.106369 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:50.106371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:50.106372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:50.106381 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:50.108142 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:50.108489 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:50.108490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:50.108490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:50.108553 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:50.108719 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:50.108720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:50.108720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:50.108862 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:50.108960 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.109048 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.109132 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:45:50.110918 29966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:45:50.111122 29966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:50.112612 30388 tablet_service.cc:1467] Processing CreateTablet for tablet c03959079b024240a001bf1de50351f6 (DEFAULT_TABLE table=test-table [id=58f115402afa41d0a4f33a0f947156ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.112612 30105 tablet_service.cc:1467] Processing CreateTablet for tablet c03959079b024240a001bf1de50351f6 (DEFAULT_TABLE table=test-table [id=58f115402afa41d0a4f33a0f947156ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.112766 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03959079b024240a001bf1de50351f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.112764 30105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03959079b024240a001bf1de50351f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.114317 30565 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Bootstrap starting.
I20260925 14:45:50.114404 30247 tablet_service.cc:1467] Processing CreateTablet for tablet c03959079b024240a001bf1de50351f6 (DEFAULT_TABLE table=test-table [id=58f115402afa41d0a4f33a0f947156ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.114526 30247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c03959079b024240a001bf1de50351f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.114753 30565 tablet_bootstrap.cc:654] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.115387 30565 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: No bootstrap required, opened a new log
I20260925 14:45:50.115458 30565 ts_tablet_manager.cc:1397] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:50.115572 30565 raft_consensus.cc:348] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.115646 30565 raft_consensus.cc:374] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.115666 30565 raft_consensus.cc:729] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.115736 30565 consensus_queue.cc:260] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.116034 30566 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Bootstrap starting.
I20260925 14:45:50.116442 30566 tablet_bootstrap.cc:654] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.116434 30565 ts_tablet_manager.cc:1428] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:50.116472 30568 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Bootstrap starting.
I20260925 14:45:50.116884 30568 tablet_bootstrap.cc:654] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.117455 30568 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: No bootstrap required, opened a new log
I20260925 14:45:50.117497 30568 ts_tablet_manager.cc:1397] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:50.117614 30568 raft_consensus.cc:348] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.117671 30568 raft_consensus.cc:374] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.117684 30568 raft_consensus.cc:729] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.117740 30568 consensus_queue.cc:260] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.117852 30568 ts_tablet_manager.cc:1428] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:50.120213 30566 tablet_bootstrap.cc:492] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: No bootstrap required, opened a new log
I20260925 14:45:50.120268 30566 ts_tablet_manager.cc:1397] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260925 14:45:50.120357 30566 raft_consensus.cc:348] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.120463 30566 raft_consensus.cc:374] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.120492 30566 raft_consensus.cc:729] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.120543 30566 consensus_queue.cc:260] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.120656 30566 ts_tablet_manager.cc:1428] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:50.155189 30488 raft_consensus.cc:482] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:50.155303 30488 raft_consensus.cc:504] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.155654 30488 leader_election.cc:288] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:50.159272 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.155862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40137 (local address 127.21.205.194:50385)
0925 14:45:50.156033 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.156044 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.156108 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.156282 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.156283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.156285 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:50.156347 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.156349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.157230 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.157230 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.158378 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.158379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.158982 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.158984 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.159046 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.159050 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.159120 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.159164 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":28,"thread_start_us":81,"threads_started":1}
I20260925 14:45:50.159345 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.155914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:42965 (local address 127.21.205.193:37669)
0925 14:45:50.155963 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.155965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.156152 (+ 187us) server_negotiation.cc:407] Connection header received
0925 14:45:50.156266 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.156267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.156276 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.156325 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:50.156506 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.156974 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.158470 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.159037 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.159113 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.159122 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.159235 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.159274 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:45:50.159507 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.155912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.21.205.194:42965)
0925 14:45:50.156079 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.156091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.156188 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.156347 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.156347 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.156349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:45:50.156395 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.156396 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.157065 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.157066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.158376 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.158379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.159118 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.159119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.159168 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.159171 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.159221 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.159252 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260925 14:45:50.159943 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.156050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:50385 (local address 127.21.205.195:40137)
0925 14:45:50.156158 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.156160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.156168 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:50.156198 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.156199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.156211 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.156271 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:45:50.156421 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.157127 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.158497 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.158874 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.158967 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.158978 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.159307 (+ 329us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.159870 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
I20260925 14:45:50.159904 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03959079b024240a001bf1de50351f6" candidate_uuid: "efcedff3810b4268a236182a7e50b0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96" is_pre_election: true
I20260925 14:45:50.160000 30126 raft_consensus.cc:2454] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efcedff3810b4268a236182a7e50b0e4 in term 0.
I20260925 14:45:50.160143 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03959079b024240a001bf1de50351f6" candidate_uuid: "efcedff3810b4268a236182a7e50b0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9" is_pre_election: true
I20260925 14:45:50.160231 30396 raft_consensus.cc:2454] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efcedff3810b4268a236182a7e50b0e4 in term 0.
I20260925 14:45:50.160210 30201 leader_election.cc:302] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:50.160480 30488 raft_consensus.cc:2781] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:50.160524 30488 raft_consensus.cc:482] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:50.160545 30488 raft_consensus.cc:3037] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.161438 30488 raft_consensus.cc:504] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.161592 30488 leader_election.cc:288] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [CANDIDATE]: Term 1 election: Requested vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:50.161733 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03959079b024240a001bf1de50351f6" candidate_uuid: "efcedff3810b4268a236182a7e50b0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96"
I20260925 14:45:50.161809 30126 raft_consensus.cc:3037] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.161762 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c03959079b024240a001bf1de50351f6" candidate_uuid: "efcedff3810b4268a236182a7e50b0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9"
I20260925 14:45:50.161876 30396 raft_consensus.cc:3037] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.162556 30126 raft_consensus.cc:2454] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efcedff3810b4268a236182a7e50b0e4 in term 1.
I20260925 14:45:50.162568 30396 raft_consensus.cc:2454] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efcedff3810b4268a236182a7e50b0e4 in term 1.
I20260925 14:45:50.162725 30201 leader_election.cc:302] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: efcedff3810b4268a236182a7e50b0e4, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:50.162812 30488 raft_consensus.cc:2781] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:50.162972 30488 raft_consensus.cc:686] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 1 LEADER]: Becoming Leader. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Running, Role: LEADER
I20260925 14:45:50.163048 30488 consensus_queue.cc:237] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.163839 29966 catalog_manager.cc:5495] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to efcedff3810b4268a236182a7e50b0e4 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "efcedff3810b4268a236182a7e50b0e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } health_report { overall_health: UNKNOWN } } }
I20260925 14:45:50.178077 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.175313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58440 (local address 127.21.205.194:41465)
0925 14:45:50.175498 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.175500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.175506 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:45:50.175546 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.175547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.175568 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.175658 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.175821 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.176350 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.177136 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.177413 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.177484 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.177669 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.177792 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.177869 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.177921 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.177942 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260925 14:45:50.179132 30396 raft_consensus.cc:1261] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Refusing update from remote peer efcedff3810b4268a236182a7e50b0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.179183 30126 raft_consensus.cc:1261] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Refusing update from remote peer efcedff3810b4268a236182a7e50b0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.179426 30574 consensus_queue.cc:1035] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.179565 30488 consensus_queue.cc:1035] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.183439 29966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.183558 29966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.184307 29966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Sending DeleteTablet for 3 replicas of tablet c03959079b024240a001bf1de50351f6
I20260925 14:45:50.184551 30105 tablet_service.cc:1514] Processing DeleteTablet for tablet c03959079b024240a001bf1de50351f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32768
I20260925 14:45:50.184653 30549 tablet_replica.cc:330] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: stopping tablet replica
I20260925 14:45:50.184696 30549 raft_consensus.cc:2229] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.184737 30549 raft_consensus.cc:2258] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.185148 30247 tablet_service.cc:1514] Processing DeleteTablet for tablet c03959079b024240a001bf1de50351f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260925 14:45:50.185232 30550 tablet_replica.cc:330] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: stopping tablet replica
I20260925 14:45:50.185276 30550 raft_consensus.cc:2229] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:50.185365 30550 raft_consensus.cc:2258] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.185703 30388 tablet_service.cc:1514] Processing DeleteTablet for tablet c03959079b024240a001bf1de50351f6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:50844
I20260925 14:45:50.185791 30551 tablet_replica.cc:330] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: stopping tablet replica
I20260925 14:45:50.185842 30551 raft_consensus.cc:2229] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.185760 29966 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 14:45:50.185897 30551 raft_consensus.cc:2258] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 14:45:50.185932 29966 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:45:50.186591 30550 ts_tablet_manager.cc:1905] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.187559 30105 tablet_service.cc:1467] Processing CreateTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c (DEFAULT_TABLE table=test-table [id=282754e447e84a1fa016e84d31822775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.187692 30105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.188190 30550 ts_tablet_manager.cc:1918] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:50.188246 30550 log.cc:1196] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/c03959079b024240a001bf1de50351f6
I20260925 14:45:50.188450 30550 ts_tablet_manager.cc:1939] T c03959079b024240a001bf1de50351f6 P efcedff3810b4268a236182a7e50b0e4: Deleting consensus metadata
I20260925 14:45:50.189270 30566 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: Bootstrap starting.
I20260925 14:45:50.189517 30551 ts_tablet_manager.cc:1905] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.189630 30566 tablet_bootstrap.cc:654] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.189613 30247 tablet_service.cc:1467] Processing CreateTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c (DEFAULT_TABLE table=test-table [id=282754e447e84a1fa016e84d31822775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.189718 30247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.189272 30549 ts_tablet_manager.cc:1905] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.191066 30551 ts_tablet_manager.cc:1918] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:50.191123 30551 log.cc:1196] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/c03959079b024240a001bf1de50351f6
I20260925 14:45:50.191150 30549 ts_tablet_manager.cc:1918] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:45:50.191206 30549 log.cc:1196] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/c03959079b024240a001bf1de50351f6
I20260925 14:45:50.191227 30388 tablet_service.cc:1467] Processing CreateTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c (DEFAULT_TABLE table=test-table [id=282754e447e84a1fa016e84d31822775]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:45:50.191349 30551 ts_tablet_manager.cc:1939] T c03959079b024240a001bf1de50351f6 P 0e6652b414014e6dab567179545ef7d9: Deleting consensus metadata
I20260925 14:45:50.191424 30549 ts_tablet_manager.cc:1939] T c03959079b024240a001bf1de50351f6 P f3bed538261d4528a6d843c0855d2b96: Deleting consensus metadata
I20260925 14:45:50.191339 30388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:50.191692 30568 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Bootstrap starting.
I20260925 14:45:50.192162 30568 tablet_bootstrap.cc:654] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.192620 29955 catalog_manager.cc:4841] TS f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669): tablet c03959079b024240a001bf1de50351f6 (table test-table [id=58f115402afa41d0a4f33a0f947156ae]) successfully deleted
I20260925 14:45:50.192775 30565 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Bootstrap starting.
I20260925 14:45:50.192932 30568 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: No bootstrap required, opened a new log
I20260925 14:45:50.192996 30568 ts_tablet_manager.cc:1397] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:50.193172 30565 tablet_bootstrap.cc:654] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:50.193164 30568 raft_consensus.cc:348] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.193246 30568 raft_consensus.cc:374] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.193269 30568 raft_consensus.cc:729] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcedff3810b4268a236182a7e50b0e4, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.193327 30568 consensus_queue.cc:260] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.193432 30568 ts_tablet_manager.cc:1428] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:50.193512 29953 catalog_manager.cc:4841] TS 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137): tablet c03959079b024240a001bf1de50351f6 (table test-table [id=58f115402afa41d0a4f33a0f947156ae]) successfully deleted
I20260925 14:45:50.194105 29953 catalog_manager.cc:4841] TS efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465): tablet c03959079b024240a001bf1de50351f6 (table test-table [id=58f115402afa41d0a4f33a0f947156ae]) successfully deleted
I20260925 14:45:50.194653 30565 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: No bootstrap required, opened a new log
I20260925 14:45:50.194705 30565 ts_tablet_manager.cc:1397] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:45:50.194790 30566 tablet_bootstrap.cc:492] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: No bootstrap required, opened a new log
I20260925 14:45:50.194844 30566 ts_tablet_manager.cc:1397] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260925 14:45:50.194811 30565 raft_consensus.cc:348] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.194876 30565 raft_consensus.cc:374] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.194896 30565 raft_consensus.cc:729] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.194936 30565 consensus_queue.cc:260] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.195016 30565 ts_tablet_manager.cc:1428] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:50.194978 30566 raft_consensus.cc:348] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.195056 30566 raft_consensus.cc:374] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:50.195077 30566 raft_consensus.cc:729] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bed538261d4528a6d843c0855d2b96, State: Initialized, Role: FOLLOWER
I20260925 14:45:50.195122 30566 consensus_queue.cc:260] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.195233 30566 ts_tablet_manager.cc:1428] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:50.434211 30491 raft_consensus.cc:482] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:50.434260 30548 raft_consensus.cc:482] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:45:50.434345 30491 raft_consensus.cc:504] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.434365 30548 raft_consensus.cc:504] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.434551 30491 leader_election.cc:288] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465)
I20260925 14:45:50.434549 30548 leader_election.cc:288] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:50.434762 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4" is_pre_election: true
I20260925 14:45:50.434783 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96" is_pre_election: true
I20260925 14:45:50.434867 30126 raft_consensus.cc:2454] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 0.
I20260925 14:45:50.434868 30267 raft_consensus.cc:2454] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 0.
I20260925 14:45:50.434890 30266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4" is_pre_election: true
I20260925 14:45:50.434906 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9" is_pre_election: true
I20260925 14:45:50.434967 30266 raft_consensus.cc:2454] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:50.434988 30396 raft_consensus.cc:2454] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3bed538261d4528a6d843c0855d2b96 in term 0.
I20260925 14:45:50.435022 30342 leader_election.cc:302] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6652b414014e6dab567179545ef7d9, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:50.435097 30491 raft_consensus.cc:2781] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:50.435137 30491 raft_consensus.cc:482] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:50.435153 30491 raft_consensus.cc:3037] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.435245 30058 leader_election.cc:302] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e6652b414014e6dab567179545ef7d9, f3bed538261d4528a6d843c0855d2b96; no voters:
I20260925 14:45:50.435376 30548 raft_consensus.cc:2781] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:45:50.435425 30548 raft_consensus.cc:482] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:45:50.435448 30548 raft_consensus.cc:3037] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.436126 30491 raft_consensus.cc:504] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.436295 30491 leader_election.cc:288] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 election: Requested vote from peers f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669), efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465)
I20260925 14:45:50.436409 30548 raft_consensus.cc:504] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.436450 30126 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3bed538261d4528a6d843c0855d2b96"
I20260925 14:45:50.436532 30548 leader_election.cc:288] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Requested vote from peers efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465), 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137)
I20260925 14:45:50.436568 30126 raft_consensus.cc:2379] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e6652b414014e6dab567179545ef7d9 in current term 1: Already voted for candidate f3bed538261d4528a6d843c0855d2b96 in this term.
I20260925 14:45:50.436537 30266 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "0e6652b414014e6dab567179545ef7d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4"
I20260925 14:45:50.436610 30266 raft_consensus.cc:3037] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:50.436672 30267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efcedff3810b4268a236182a7e50b0e4"
I20260925 14:45:50.436700 30396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f49e042a7dd4d3b90f5ba0802cafd1c" candidate_uuid: "f3bed538261d4528a6d843c0855d2b96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e6652b414014e6dab567179545ef7d9"
I20260925 14:45:50.436780 30396 raft_consensus.cc:2379] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3bed538261d4528a6d843c0855d2b96 in current term 1: Already voted for candidate 0e6652b414014e6dab567179545ef7d9 in this term.
I20260925 14:45:50.436957 30058 leader_election.cc:302] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3bed538261d4528a6d843c0855d2b96; no voters: 0e6652b414014e6dab567179545ef7d9, efcedff3810b4268a236182a7e50b0e4
I20260925 14:45:50.437104 30548 raft_consensus.cc:2726] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 14:45:50.437453 30266 raft_consensus.cc:2454] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e6652b414014e6dab567179545ef7d9 in term 1.
I20260925 14:45:50.437618 30340 leader_election.cc:302] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e6652b414014e6dab567179545ef7d9, efcedff3810b4268a236182a7e50b0e4; no voters: f3bed538261d4528a6d843c0855d2b96
I20260925 14:45:50.437714 30491 raft_consensus.cc:2781] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:50.437830 30491 raft_consensus.cc:686] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 LEADER]: Becoming Leader. State: Replica: 0e6652b414014e6dab567179545ef7d9, State: Running, Role: LEADER
I20260925 14:45:50.437875 30491 consensus_queue.cc:237] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } }
I20260925 14:45:50.438617 29966 catalog_manager.cc:5495] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e6652b414014e6dab567179545ef7d9 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "0e6652b414014e6dab567179545ef7d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e6652b414014e6dab567179545ef7d9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40137 } health_report { overall_health: HEALTHY } } }
I20260925 14:45:50.484899 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.482412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50862 (local address 127.21.205.195:40137)
0925 14:45:50.482472 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.482478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.482485 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:50.482505 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.482506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.482522 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.482583 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.482695 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.483359 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.484219 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.484445 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.484520 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.484648 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.484685 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.484731 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.484788 (+ 57us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.484811 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:45:50.489941 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.487418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.21.205.195:33391)
0925 14:45:50.487564 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.487575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.487634 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.487762 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.487763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.487764 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:45:50.487867 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.487868 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.488469 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.488470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.489088 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.489090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.489506 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.489507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.489564 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.489580 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.489796 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.489798 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.489836 (+ 38us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.489876 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":85,"threads_started":1}
I20260925 14:45:50.489940 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.487527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33391 (local address 127.21.205.193:37669)
0925 14:45:50.487589 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.487591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.487629 (+ 38us) server_negotiation.cc:407] Connection header received
0925 14:45:50.487690 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.487690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.487703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.487751 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.487929 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.488405 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.489208 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.489434 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.489490 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.489691 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.489725 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.489784 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.489839 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.489865 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:45:50.492455 30126 raft_consensus.cc:1261] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Refusing update from remote peer 0e6652b414014e6dab567179545ef7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.492439 30266 raft_consensus.cc:1261] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Refusing update from remote peer 0e6652b414014e6dab567179545ef7d9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:45:50.492686 30491 consensus_queue.cc:1035] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efcedff3810b4268a236182a7e50b0e4" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.492817 30584 consensus_queue.cc:1035] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3bed538261d4528a6d843c0855d2b96" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:45:50.499648 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.497244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:50023 (local address 127.21.205.195:40137)
0925 14:45:50.497356 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:50.497358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:50.497367 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:50.497389 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:50.497390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:50.497402 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:50.497456 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:45:50.497579 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.498048 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.498751 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.499039 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.499107 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.499348 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.499397 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.499452 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:50.499528 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 14:45:50.499566 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 14:45:50.499648 30546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.497198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40137 (local address 127.21.205.193:50023)
0925 14:45:50.497247 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.497258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:50.497296 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:50.497476 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:50.497477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:50.497479 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 14:45:50.497527 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.497529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.498114 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.498115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.498663 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:50.498665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:50.499143 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:50.499144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:50.499205 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:50.499214 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 14:45:50.499464 (+ 250us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 14:45:50.499467 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:45:50.499522 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:45:50.499589 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 14:45:50.505607 29966 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.505712 29966 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:45:50.506263 29966 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d44693497bf045c6829ccf1e7b0649a4: Sending DeleteTablet for 3 replicas of tablet 3f49e042a7dd4d3b90f5ba0802cafd1c
I20260925 14:45:50.506511 30105 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:32768
I20260925 14:45:50.506558 30388 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:50844
I20260925 14:45:50.506546 30247 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:45:50 UTC) from {username='oryx', principal='oryx/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:58434
I20260925 14:45:50.506649 30550 tablet_replica.cc:330] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: stopping tablet replica
I20260925 14:45:50.506656 30549 tablet_replica.cc:330] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96: stopping tablet replica
I20260925 14:45:50.506697 30550 raft_consensus.cc:2229] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.506714 30549 raft_consensus.cc:2229] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:45:50.506735 30550 raft_consensus.cc:2258] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.506753 30549 raft_consensus.cc:2258] T 3f49e042a7dd4d3b90f5ba0802cafd1c P f3bed538261d4528a6d843c0855d2b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.506825 30551 tablet_replica.cc:330] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: stopping tablet replica
I20260925 14:45:50.506896 30551 raft_consensus.cc:2229] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:45:50.506960 30551 raft_consensus.cc:2258] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:45:50.507026 30550 ts_tablet_manager.cc:1905] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.507200 30551 ts_tablet_manager.cc:1905] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:45:50.507493 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30041
I20260925 14:45:50.508706 30550 ts_tablet_manager.cc:1918] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:50.508749 30550 log.cc:1196] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/3f49e042a7dd4d3b90f5ba0802cafd1c
I20260925 14:45:50.508906 30550 ts_tablet_manager.cc:1939] T 3f49e042a7dd4d3b90f5ba0802cafd1c P efcedff3810b4268a236182a7e50b0e4: Deleting consensus metadata
I20260925 14:45:50.509069 30551 ts_tablet_manager.cc:1918] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 14:45:50.509153 30551 log.cc:1196] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/3f49e042a7dd4d3b90f5ba0802cafd1c
I20260925 14:45:50.509385 30551 ts_tablet_manager.cc:1939] T 3f49e042a7dd4d3b90f5ba0802cafd1c P 0e6652b414014e6dab567179545ef7d9: Deleting consensus metadata
I20260925 14:45:50.509375 29953 catalog_manager.cc:4841] TS efcedff3810b4268a236182a7e50b0e4 (127.21.205.194:41465): tablet 3f49e042a7dd4d3b90f5ba0802cafd1c (table test-table [id=282754e447e84a1fa016e84d31822775]) successfully deleted
I20260925 14:45:50.509706 29953 catalog_manager.cc:4841] TS 0e6652b414014e6dab567179545ef7d9 (127.21.205.195:40137): tablet 3f49e042a7dd4d3b90f5ba0802cafd1c (table test-table [id=282754e447e84a1fa016e84d31822775]) successfully deleted
W20260925 14:45:50.515938 29955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:37669 (error 108)
I20260925 14:45:50.516140 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30182
I20260925 14:45:50.517596 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:50.516394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37669 (local address 127.0.0.1:32786)
0925 14:45:50.517480 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:50.517534 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1053,"thread_start_us":43,"threads_started":1}
W20260925 14:45:50.517716 29955 catalog_manager.cc:4584] TS f3bed538261d4528a6d843c0855d2b96 (127.21.205.193:37669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f49e042a7dd4d3b90f5ba0802cafd1c: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37669: connect: Connection refused (error 111)
I20260925 14:45:50.522643 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30323
I20260925 14:45:50.528817 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 29936
2026-09-25T14:45:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3058 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[30597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[30597](info): set up 2 sockets
Sep 25 14:45:50 dist-test-slave-d7lw krb5kdc[30597](info): commencing operation
krb5kdc: starting...
W20260925 14:45:51.579185 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:51 dist-test-slave-d7lw krb5kdc[30597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:51.702458 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38253
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:38253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:51.787432 30610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:51.787670 30610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:51.787698 30610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:51.788975 30610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:51.789029 30610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:51.789044 30610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:51.789083 30610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:51.789114 30610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:51.790742 30610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:38253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:51.791040 30610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:51.791285 30610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:51.796241 30616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:51.796290 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:51.796409 30610 server_base.cc:1034] running on GCE node
W20260925 14:45:51.796293 30615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:51.796784 30610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:51.797122 30610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:51.798319 30610 hybrid_clock.cc:648] HybridClock initialized: now 1790347551798293 us; error 48 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T14:45:51Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1269 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:45:51 dist-test-slave-d7lw krb5kdc[30623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:45:51 dist-test-slave-d7lw krb5kdc[30623](info): set up 2 sockets
Sep 25 14:45:51 dist-test-slave-d7lw krb5kdc[30623](info): commencing operation
krb5kdc: starting...
W20260925 14:45:52.847601 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:45:52 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:45:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:45:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:45:52.968973 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35071
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:53.056196 30636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:53.056469 30636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:53.056512 30636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:53.057822 30636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:53.057904 30636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:53.057926 30636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:53.057948 30636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:53.057976 30636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:53.059648 30636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:53.060003 30636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:53.060256 30636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:53.065565 30642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.065750 30636 server_base.cc:1034] running on GCE node
W20260925 14:45:53.065580 30641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.065598 30644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.066222 30636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:53.066556 30636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:53.067741 30636 hybrid_clock.cc:648] HybridClock initialized: now 1790347553067728 us; error 31 us; skew 500 ppm
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:53.070497 30636 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:53.071027 30636 webserver.cc:460] Webserver started at http://127.21.205.254:41045/ using document root <none> and password file <none>
I20260925 14:45:53.071236 30636 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:53.071281 30636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:53.071380 30636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:53.072547 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "d7a741f3436d4439b3479e73310bb414"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "1a99f4e78790f716e3a05f47b754a0f9"
server_key_iv: "5664f4418e550f02a7b9bba69d9c029b"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.072892 30636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "d7a741f3436d4439b3479e73310bb414"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "1a99f4e78790f716e3a05f47b754a0f9"
server_key_iv: "5664f4418e550f02a7b9bba69d9c029b"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.074306 30636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:53.075042 30651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.075258 30636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:53.075374 30636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "d7a741f3436d4439b3479e73310bb414"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "1a99f4e78790f716e3a05f47b754a0f9"
server_key_iv: "5664f4418e550f02a7b9bba69d9c029b"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.075469 30636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:53.099345 30636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:53.101483 30636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:53.101649 30636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:53.105732 30636 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:35071
I20260925 14:45:53.105804 30704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:35071 every 8 connection(s)
I20260925 14:45:53.106158 30636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:53.107131 30705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:45:53.109563 30705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Bootstrap starting.
I20260925 14:45:53.110152 30705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Neither blocks nor log segments found. Creating new log.
I20260925 14:45:53.110400 30705 log.cc:824] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:53.111121 30705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: No bootstrap required, opened a new log
I20260925 14:45:53.112761 30705 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.112915 30705 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:45:53.112938 30705 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a741f3436d4439b3479e73310bb414, State: Initialized, Role: FOLLOWER
I20260925 14:45:53.113108 30705 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.113188 30705 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:53.113212 30705 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:53.113247 30705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:45:53.113777 30705 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.113868 30705 leader_election.cc:302] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7a741f3436d4439b3479e73310bb414; no voters:
I20260925 14:45:53.114013 30705 leader_election.cc:288] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:45:53.114085 30708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:45:53.114292 30708 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 LEADER]: Becoming Leader. State: Replica: d7a741f3436d4439b3479e73310bb414, State: Running, Role: LEADER
I20260925 14:45:53.114300 30705 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:53.114362 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30636
I20260925 14:45:53.114418 30708 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.114464 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:45:53.114679 22327 external_mini_cluster.cc:1428] Setting key 30b3decdadbadd3cc98a756d9d7e8ad3
I20260925 14:45:53.114916 30709 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7a741f3436d4439b3479e73310bb414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } } }
I20260925 14:45:53.114997 30709 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:53.115540 30710 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7a741f3436d4439b3479e73310bb414. Latest consensus state: current_term: 1 leader_uuid: "d7a741f3436d4439b3479e73310bb414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } } }
I20260925 14:45:53.115612 30710 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: This master's current role is: LEADER
W20260925 14:45:53.116315 30727 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:45:53.116379 30727 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:45:53.116482 30717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:53.116700 30717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:53.118345 30717 catalog_manager.cc:1347] Generated new cluster ID: 668f3fe86b554fbfa5ff97d456e63d35
I20260925 14:45:53.118403 30717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:53.127776 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.115346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60362 (local address 127.21.205.254:35071)
0925 14:45:53.115527 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:53.115540 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:53.116111 (+ 571us) server_negotiation.cc:407] Connection header received
0925 14:45:53.116262 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:53.116269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:53.116323 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:53.116486 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:53.117403 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.118830 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.119996 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.120718 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.120837 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.123828 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:53.123834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:53.123834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:53.123850 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:53.125945 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.126407 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.126410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.126410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.126542 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.126771 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.126772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.126772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.127100 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:53.127248 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:53.127377 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 14:45:53.127521 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260925 14:45:53.133276 30717 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:45:53.134074 30717 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:53.141088 30717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Generated new TSK 0
I20260925 14:45:53.141332 30717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:45:53.170684 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:53.274734 30732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:53.275024 30732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:53.275069 30732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:53.276355 30732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:53.276432 30732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:53.276482 30732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:53.278044 30732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:53.278362 30732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:53.278611 30732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:45:53.284039 30732 server_base.cc:1034] running on GCE node
W20260925 14:45:53.284013 30738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.284000 30740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.284020 30737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.284386 30732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:53.284669 30732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:53.285816 30732 hybrid_clock.cc:648] HybridClock initialized: now 1790347553285802 us; error 28 us; skew 500 ppm
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:53.288470 30732 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:45:53.288925 30732 webserver.cc:460] Webserver started at http://127.21.205.193:38451/ using document root <none> and password file <none>
I20260925 14:45:53.289129 30732 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:53.289171 30732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:53.289268 30732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:53.290354 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "d62940c9c84c48658e161d8a382c8e35"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "f52a82251a05d313bdd596883e7af6b7"
server_key_iv: "c1490c9f5285914fc4a435a8e3c96259"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.290680 30732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "d62940c9c84c48658e161d8a382c8e35"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "f52a82251a05d313bdd596883e7af6b7"
server_key_iv: "c1490c9f5285914fc4a435a8e3c96259"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.292138 30732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:45:53.292928 30747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.293123 30732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:53.293232 30732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "d62940c9c84c48658e161d8a382c8e35"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "f52a82251a05d313bdd596883e7af6b7"
server_key_iv: "c1490c9f5285914fc4a435a8e3c96259"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.293316 30732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:53.316560 30732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:53.319031 30732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:53.319206 30732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:53.319455 30732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:53.319869 30732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:53.319926 30732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.319970 30732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:53.319998 30732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.325837 30732 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:44671
I20260925 14:45:53.325917 30861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:44671 every 8 connection(s)
I20260925 14:45:53.326273 30732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:45:53.326452 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30732
I20260925 14:45:53.326584 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:45:53.326820 22327 external_mini_cluster.cc:1428] Setting key df00a80f302ff93997ffbca21450dc9d
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
I20260925 14:45:53.337615 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.327182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:49759 (local address 127.21.205.254:35071)
0925 14:45:53.327246 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:53.327248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:53.327486 (+ 238us) server_negotiation.cc:407] Connection header received
0925 14:45:53.328110 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:53.328111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:53.328128 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:53.328202 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:53.329146 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.329989 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.331046 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.331270 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.331359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.333860 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:53.333862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:53.333862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:53.333871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:53.335565 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.336199 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.336200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.336201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.336273 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.336614 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.336616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.336616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.336802 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:53.336891 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:53.337311 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 14:45:53.337471 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:45:53.337755 30864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.327177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:49759)
0925 14:45:53.327350 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:53.327393 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:53.328012 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:53.328262 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:53.328265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:53.328845 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:53.329049 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.329055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.330093 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.330093 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.330939 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.330941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.331377 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.331378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.331517 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.332110 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:53.332120 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:53.333749 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:53.335676 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.335686 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.335689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.336091 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.336367 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.336368 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.336368 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.336532 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.336895 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:53.336902 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:53.337105 (+ 203us) client_negotiation.cc:769] Sending connection context
0925 14:45:53.337228 (+ 123us) client_negotiation.cc:240] Negotiation successful
0925 14:45:53.337459 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
I20260925 14:45:53.339046 30862 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35071
I20260925 14:45:53.339179 30862 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:53.339432 30862 heartbeater.cc:511] Master 127.21.205.254:35071 requested a full tablet report, sending...
I20260925 14:45:53.342303 30669 ts_manager.cc:194] Registered new tserver with Master: d62940c9c84c48658e161d8a382c8e35 (127.21.205.193:44671)
I20260925 14:45:53.343348 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:49759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:45:53.354952 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:53.440171 30867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:53.440415 30867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:53.440443 30867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:53.441704 30867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:53.441752 30867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:53.441783 30867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:53.443413 30867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:53.443652 30867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:53.443917 30867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:53.449213 30873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.449244 30872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.449334 30867 server_base.cc:1034] running on GCE node
W20260925 14:45:53.449218 30875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.449741 30867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:53.450068 30867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:53.451239 30867 hybrid_clock.cc:648] HybridClock initialized: now 1790347553451217 us; error 39 us; skew 500 ppm
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:53.454128 30867 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:45:53.454667 30867 webserver.cc:460] Webserver started at http://127.21.205.194:36525/ using document root <none> and password file <none>
I20260925 14:45:53.454900 30867 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:53.454947 30867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:53.455057 30867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:53.456313 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "bd4f04a4fe8e40039bac8700b212c8a0"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "0d2833c38af2a6c3287f09e9436e2cec"
server_key_iv: "dcfa6b51839186e51ecec1fbc8d798f0"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.456660 30867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "bd4f04a4fe8e40039bac8700b212c8a0"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "0d2833c38af2a6c3287f09e9436e2cec"
server_key_iv: "dcfa6b51839186e51ecec1fbc8d798f0"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.458143 30867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:45:53.458986 30882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.459182 30867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:53.459303 30867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "bd4f04a4fe8e40039bac8700b212c8a0"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "0d2833c38af2a6c3287f09e9436e2cec"
server_key_iv: "dcfa6b51839186e51ecec1fbc8d798f0"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.459393 30867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:53.482312 30867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:53.485045 30867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:53.485250 30867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:53.485526 30867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:53.485884 30867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:53.485939 30867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.485986 30867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:53.486018 30867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.492928 30867 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38097
I20260925 14:45:53.492993 30996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38097 every 8 connection(s)
I20260925 14:45:53.493372 30867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:53.500403 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 30867
I20260925 14:45:53.500540 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:53.500784 22327 external_mini_cluster.cc:1428] Setting key 270219e9a0d88ce9025523c3694406c6
I20260925 14:45:53.504568 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.494529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:57073 (local address 127.21.205.254:35071)
0925 14:45:53.494609 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:53.494612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:53.494812 (+ 200us) server_negotiation.cc:407] Connection header received
0925 14:45:53.495375 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:53.495376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:53.495386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:53.495433 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:53.496626 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.497131 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.498285 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.498481 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.498538 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.501336 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:53.501338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:53.501338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:53.501345 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:53.502875 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.503401 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.503402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.503403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.503477 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.503718 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.503719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.503719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.503892 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:53.503973 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:53.504308 (+ 335us) server_negotiation.cc:299] Negotiation successful
0925 14:45:53.504413 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:45:53.504703 30999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.494501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:57073)
0925 14:45:53.494683 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:53.494726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:53.495255 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:53.495494 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:53.495498 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:53.496240 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:53.496526 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.496533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.497218 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.497219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.498198 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.498200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.498562 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.498563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.498679 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.499321 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:53.499331 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:53.501242 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:53.502985 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.502997 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.503000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.503324 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.503545 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.503546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.503546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.503657 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.504000 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:53.504008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:53.504156 (+ 148us) client_negotiation.cc:769] Sending connection context
0925 14:45:53.504269 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 14:45:53.504436 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260925 14:45:53.505210 30997 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35071
I20260925 14:45:53.505335 30997 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:53.505564 30997 heartbeater.cc:511] Master 127.21.205.254:35071 requested a full tablet report, sending...
I20260925 14:45:53.506042 30669 ts_manager.cc:194] Registered new tserver with Master: bd4f04a4fe8e40039bac8700b212c8a0 (127.21.205.194:38097)
I20260925 14:45:53.506616 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:57073
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:45:53.526930 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:53.612803 31002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:53.613075 31002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:53.613106 31002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:53.614580 31002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:53.614677 31002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:53.614729 31002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:53.616708 31002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:53.617030 31002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:53.617494 31002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:53.623580 31010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.623584 31007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.623690 31002 server_base.cc:1034] running on GCE node
W20260925 14:45:53.623631 31008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.624070 31002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:53.624358 31002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:53.625557 31002 hybrid_clock.cc:648] HybridClock initialized: now 1790347553625535 us; error 38 us; skew 500 ppm
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:53.628530 31002 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:45:53.629030 31002 webserver.cc:460] Webserver started at http://127.21.205.195:35559/ using document root <none> and password file <none>
I20260925 14:45:53.629293 31002 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:53.629364 31002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:53.629515 31002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:45:53.630682 31002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "54f330128d624af19c1729ffabec5493"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "d73822e937437cb99f49741d580d7e4f"
server_key_iv: "c90826c54709a6f34994d6fa995b5bf1"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.631088 31002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "54f330128d624af19c1729ffabec5493"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "d73822e937437cb99f49741d580d7e4f"
server_key_iv: "c90826c54709a6f34994d6fa995b5bf1"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.632596 31002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:45:53.633425 31017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.633738 31002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:53.633824 31002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "54f330128d624af19c1729ffabec5493"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "d73822e937437cb99f49741d580d7e4f"
server_key_iv: "c90826c54709a6f34994d6fa995b5bf1"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.633934 31002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:53.665604 31002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:53.667642 31002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:53.667826 31002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:53.668084 31002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:53.668442 31002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:53.668494 31002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.668538 31002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:53.668565 31002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.674491 31002 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:43209
I20260925 14:45:53.674571 31131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:43209 every 8 connection(s)
I20260925 14:45:53.674923 31002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:45:53.682770 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31002
I20260925 14:45:53.682909 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:45:53.683212 22327 external_mini_cluster.cc:1428] Setting key fd1208c31d695693b5635e3772275465
I20260925 14:45:53.685781 30722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.675679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:35007 (local address 127.21.205.254:35071)
0925 14:45:53.675749 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:53.675752 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:53.676023 (+ 271us) server_negotiation.cc:407] Connection header received
0925 14:45:53.676527 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:53.676528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:53.676538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:53.676593 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:53.677604 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.678137 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.679239 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.679437 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.679494 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.681845 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:53.681847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:53.681848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:53.681856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:53.683711 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.684516 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.684518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.684518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.684594 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:53.684909 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:53.684910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:53.684911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:53.685048 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:53.685116 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:53.685626 (+ 510us) server_negotiation.cc:299] Negotiation successful
0925 14:45:53.685710 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:45:53.686101 31134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:53.675677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:35007)
0925 14:45:53.675895 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:53.675935 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:53.676438 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:53.676668 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:53.676672 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:53.677301 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:53.677519 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.677524 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.678251 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.678251 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.679166 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:53.679170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:53.679517 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:53.679517 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:53.679627 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:53.680241 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:53.680250 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:53.681730 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:45:53.683956 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.683969 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.683973 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.684416 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.684655 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:45:53.684656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:45:53.684656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:45:53.684771 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:45:53.685113 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:45:53.685118 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:45:53.685252 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:45:53.685341 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 14:45:53.685773 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":122,"threads_started":1}
I20260925 14:45:53.686687 31132 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35071
I20260925 14:45:53.686801 31132 heartbeater.cc:464] Registering TS with master...
I20260925 14:45:53.687007 31132 heartbeater.cc:511] Master 127.21.205.254:35071 requested a full tablet report, sending...
I20260925 14:45:53.687453 30669 ts_manager.cc:194] Registered new tserver with Master: 54f330128d624af19c1729ffabec5493 (127.21.205.195:43209)
I20260925 14:45:53.687969 30669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:35007
I20260925 14:45:53.695524 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
Entry for principal oryx/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab.
WARNING: no policy specified for oryx/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
Entry for principal oryx/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab.
WARNING: no policy specified for oryx/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
Entry for principal oryx/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab.
I20260925 14:45:53.771045 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30732
I20260925 14:45:53.776909 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30867
I20260925 14:45:53.781658 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31002
I20260925 14:45:53.786449 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 30636
I20260925 14:45:53.792304 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35071
--webserver_interface=127.21.205.254
--webserver_port=41045
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35071
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:45:53.876572 31141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:53.876869 31141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:53.876914 31141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:53.878209 31141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:45:53.878300 31141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:53.878326 31141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:53.878348 31141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:45:53.878396 31141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:45:53.880118 31141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=41045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:53.880456 31141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:53.880707 31141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:53.885993 31147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.886117 31141 server_base.cc:1034] running on GCE node
W20260925 14:45:53.885986 31146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:53.886217 31149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:53.886515 31141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:53.886855 31141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:53.888007 31141 hybrid_clock.cc:648] HybridClock initialized: now 1790347553887994 us; error 31 us; skew 500 ppm
Sep 25 14:45:53 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347553, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:53.902230 31141 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:45:53.902838 31141 webserver.cc:460] Webserver started at http://127.21.205.254:41045/ using document root <none> and password file <none>
I20260925 14:45:53.903081 31141 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:53.903146 31141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:53.904891 31141 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:53.905659 31156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:53.905874 31141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:45:53.905959 31141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "d7a741f3436d4439b3479e73310bb414"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "1a99f4e78790f716e3a05f47b754a0f9"
server_key_iv: "5664f4418e550f02a7b9bba69d9c029b"
server_key_version: "encryptionkey@0"
I20260925 14:45:53.906379 31141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:53.924484 31141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:53.926491 31141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:53.926626 31141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:53.931301 31141 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:35071
I20260925 14:45:53.931336 31208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:35071 every 8 connection(s)
I20260925 14:45:53.931865 31141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:45:53.933521 31209 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:45:53.934173 31209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Bootstrap starting.
I20260925 14:45:53.937903 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31141
I20260925 14:45:53.938235 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:44671
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:35071
--webserver_port=38451
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:45:53.943137 31209 log.cc:824] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Log is configured to *not* fsync() on all Append() calls
I20260925 14:45:53.945152 31209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:45:53.945465 31209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Bootstrap complete.
I20260925 14:45:53.947990 31209 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.948261 31209 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a741f3436d4439b3479e73310bb414, State: Initialized, Role: FOLLOWER
I20260925 14:45:53.948468 31209 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.948562 31209 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:45:53.948617 31209 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:45:53.948668 31209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 1 FOLLOWER]: Advancing to term 2
I20260925 14:45:53.949745 31209 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.949880 31209 leader_election.cc:302] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7a741f3436d4439b3479e73310bb414; no voters:
I20260925 14:45:53.950074 31209 leader_election.cc:288] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 14:45:53.950152 31213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 14:45:53.950356 31213 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [term 2 LEADER]: Becoming Leader. State: Replica: d7a741f3436d4439b3479e73310bb414, State: Running, Role: LEADER
I20260925 14:45:53.950361 31209 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:45:53.950873 31213 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } }
I20260925 14:45:53.951237 31214 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d7a741f3436d4439b3479e73310bb414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } } }
I20260925 14:45:53.951251 31215 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7a741f3436d4439b3479e73310bb414. Latest consensus state: current_term: 2 leader_uuid: "d7a741f3436d4439b3479e73310bb414" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7a741f3436d4439b3479e73310bb414" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35071 } } }
I20260925 14:45:53.951436 31215 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:53.951377 31214 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414 [sys.catalog]: This master's current role is: LEADER
I20260925 14:45:53.951741 31224 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:45:53.952476 31224 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:45:53.952620 31224 catalog_manager.cc:1259] Loaded cluster ID: 668f3fe86b554fbfa5ff97d456e63d35
I20260925 14:45:53.952654 31224 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:45:53.953898 31224 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:45:53.954107 31224 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d7a741f3436d4439b3479e73310bb414: Loaded TSK: 0
I20260925 14:45:53.954344 31224 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:45:54.031121 31211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:54.031349 31211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:54.031376 31211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:54.032716 31211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:54.032764 31211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:54.032795 31211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:45:54.034448 31211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.193.keytab
--principal=oryx/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=38451
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:54.034704 31211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:54.034982 31211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:54.040251 31235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.040249 31238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.040249 31236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:54.041029 31211 server_base.cc:1034] running on GCE node
I20260925 14:45:54.041256 31211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:54.041531 31211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:54.042695 31211 hybrid_clock.cc:648] HybridClock initialized: now 1790347554042679 us; error 32 us; skew 500 ppm
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:54.045608 31211 init.cc:377] Logged in from keytab as oryx/127.21.205.193@KRBTEST.COM (short username oryx)
I20260925 14:45:54.046170 31211 webserver.cc:460] Webserver started at http://127.21.205.193:38451/ using document root <none> and password file <none>
I20260925 14:45:54.046481 31211 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:54.046581 31211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:54.048095 31211 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:45:54.048739 31245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.048909 31211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:45:54.048995 31211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "d62940c9c84c48658e161d8a382c8e35"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "f52a82251a05d313bdd596883e7af6b7"
server_key_iv: "c1490c9f5285914fc4a435a8e3c96259"
server_key_version: "encryptionkey@0"
I20260925 14:45:54.049379 31211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:54.058071 31211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:54.060766 31211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:54.060931 31211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:54.061215 31211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:54.061568 31211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:54.061604 31211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.061626 31211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:54.061656 31211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.069082 31211 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:44671
I20260925 14:45:54.069231 31360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:44671 every 8 connection(s)
I20260925 14:45:54.069568 31211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:45:54.075069 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31211
I20260925 14:45:54.075438 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:38097
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:35071
--webserver_port=36525
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.080556 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.070428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:43587)
0925 14:45:54.070651 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.070706 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.071351 (+ 645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.071916 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.071921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.072729 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.073055 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.073063 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.074269 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.074270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.075575 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.075578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.076507 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.076508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.076686 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.077343 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.077352 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.080275 (+ 2923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":92,"threads_started":1}
W20260925 14:45:54.080695 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.070563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:43587 (local address 127.21.205.254:35071)
0925 14:45:54.070746 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.070753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.070786 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:45:54.071591 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.071616 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.071664 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.071840 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.073200 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.074148 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.075690 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.076393 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.076515 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.080478 (+ 3963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:43587: BlockingRecv error: recv got EOF from 127.21.205.193:43587 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
W20260925 14:45:54.081154 31361 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:35071 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.205.254:35071: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.087687 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.081431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:57689)
0925 14:45:54.081484 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.081496 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.081541 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.081708 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.081709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.081898 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.081978 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.081981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.083049 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.083050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.084104 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.084107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.084634 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.084635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.084733 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.085284 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.085291 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.087598 (+ 2307us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 14:45:54.087821 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.081477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:57689 (local address 127.21.205.254:35071)
0925 14:45:54.081519 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.081521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.081531 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:45:54.081619 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.081620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.081632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.081685 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.082089 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.082936 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.084203 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.084539 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.084622 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.087726 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:57689: BlockingRecv error: recv got EOF from 127.21.205.193:57689 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.093883 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.087998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:39215)
0925 14:45:54.088230 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.088240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.088286 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.088439 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.088440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.088589 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.088633 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.088635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.089556 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.089557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.090381 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.090385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.090914 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.090915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.090971 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.091469 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.091474 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.093822 (+ 2348us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":83,"threads_started":1}
W20260925 14:45:54.094046 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.088180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:39215 (local address 127.21.205.254:35071)
0925 14:45:54.088242 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.088244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.088272 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:45:54.088350 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.088351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.088362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.088422 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.088700 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.089454 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.090486 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.090782 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.090822 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.093957 (+ 3135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:39215: BlockingRecv error: recv got EOF from 127.21.205.193:39215 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260925 14:45:54.094058 31361 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:45:54.165431 31365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:54.165678 31365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:54.165704 31365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:54.167052 31365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:54.167107 31365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:54.167138 31365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:45:54.168792 31365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.194.keytab
--principal=oryx/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:38097
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=36525
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:54.169086 31365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:54.169333 31365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:54.174726 31374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.174726 31372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.174731 31371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:54.175009 31365 server_base.cc:1034] running on GCE node
I20260925 14:45:54.175235 31365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:54.175532 31365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:54.176673 31365 hybrid_clock.cc:648] HybridClock initialized: now 1790347554176662 us; error 30 us; skew 500 ppm
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:54.179229 31365 init.cc:377] Logged in from keytab as oryx/127.21.205.194@KRBTEST.COM (short username oryx)
I20260925 14:45:54.179718 31365 webserver.cc:460] Webserver started at http://127.21.205.194:36525/ using document root <none> and password file <none>
I20260925 14:45:54.179999 31365 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:54.180064 31365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:54.181427 31365 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:54.181953 31381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.182142 31365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:45:54.182238 31365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "bd4f04a4fe8e40039bac8700b212c8a0"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "0d2833c38af2a6c3287f09e9436e2cec"
server_key_iv: "dcfa6b51839186e51ecec1fbc8d798f0"
server_key_version: "encryptionkey@0"
I20260925 14:45:54.182629 31365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:54.210740 31365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:54.213312 31365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:54.213449 31365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:54.213647 31365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:54.214000 31365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:54.214040 31365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.214082 31365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:54.214112 31365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.219836 31365 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38097
I20260925 14:45:54.219944 31496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38097 every 8 connection(s)
I20260925 14:45:54.220304 31365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:45:54.221601 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31365
I20260925 14:45:54.221897 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:43209
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:35071
--webserver_port=35559
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 14:45:54.228775 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.221071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:33803 (local address 127.21.205.254:35071)
0925 14:45:54.221140 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.221142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.221447 (+ 305us) server_negotiation.cc:407] Connection header received
0925 14:45:54.222063 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.222065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.222083 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.222114 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.223239 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.223738 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.224862 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.225168 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.225241 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.228722 (+ 3481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:33803: BlockingRecv error: recv got EOF from 127.21.205.194:33803 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:45:54.228910 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.221083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:33803)
0925 14:45:54.221292 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.221332 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.221908 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.222284 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.222288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.222897 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.223126 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.223132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.223900 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.223902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.224740 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.224742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.225264 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.225265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.225412 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.226000 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.226009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.228623 (+ 2614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":114,"threads_started":1}
W20260925 14:45:54.229159 31497 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:35071 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.205.254:35071: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.233903 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.229423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:38481)
0925 14:45:54.229473 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.229484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.229541 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.229678 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.229678 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.229822 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.229882 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.229884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.230543 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.230544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.231070 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.231072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.231430 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.231431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.231503 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.232068 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.232073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.233815 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 14:45:54.233986 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.229486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:38481 (local address 127.21.205.254:35071)
0925 14:45:54.229546 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.229548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.229567 (+ 19us) server_negotiation.cc:407] Connection header received
0925 14:45:54.229600 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.229601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.229611 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.229673 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.229959 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.230463 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.231156 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.231351 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.231431 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.233912 (+ 2481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:38481: BlockingRecv error: recv got EOF from 127.21.205.194:38481 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.239122 31499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.234298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:50603)
0925 14:45:54.234346 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.234356 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.234393 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.234507 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.234507 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.234654 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.234699 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.234700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.235310 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.235311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.236194 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.236197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.236611 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.236612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.236676 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.237144 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.237149 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.239025 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 14:45:54.239176 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.234352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:50603 (local address 127.21.205.254:35071)
0925 14:45:54.234407 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.234409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.234416 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:45:54.234441 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.234441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.234449 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.234492 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.234761 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.235233 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.236280 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.236530 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.236609 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.239111 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:50603: BlockingRecv error: recv got EOF from 127.21.205.194:50603 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260925 14:45:54.239280 31497 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:45:54.309525 31500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:45:54.309757 31500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:45:54.309783 31500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:45:54.311120 31500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:45:54.311174 31500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:45:54.311204 31500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:45:54.312798 31500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/krb5kdc/oryx_127.21.205.195.keytab
--principal=oryx/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:43209
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=35559
--tserver_master_addrs=127.21.205.254:35071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:45:54.313113 31500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:45:54.313381 31500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:45:54.318696 31508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.318681 31505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:45:54.318684 31506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:45:54.318980 31500 server_base.cc:1034] running on GCE node
I20260925 14:45:54.319207 31500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:45:54.319499 31500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:45:54.320678 31500 hybrid_clock.cc:648] HybridClock initialized: now 1790347554320670 us; error 37 us; skew 500 ppm
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347554, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:45:54.323367 31500 init.cc:377] Logged in from keytab as oryx/127.21.205.195@KRBTEST.COM (short username oryx)
I20260925 14:45:54.323864 31500 webserver.cc:460] Webserver started at http://127.21.205.195:35559/ using document root <none> and password file <none>
I20260925 14:45:54.324076 31500 fs_manager.cc:359] Metadata directory not provided
I20260925 14:45:54.324131 31500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:45:54.325585 31500 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:54.326195 31515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.326385 31500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:45:54.326486 31500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "54f330128d624af19c1729ffabec5493"
format_stamp: "Formatted at 2026-09-25 14:45:53 on dist-test-slave-d7lw"
server_key: "d73822e937437cb99f49741d580d7e4f"
server_key_iv: "c90826c54709a6f34994d6fa995b5bf1"
server_key_version: "encryptionkey@0"
I20260925 14:45:54.326879 31500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:45:54.362473 31500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:45:54.365115 31500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:45:54.365249 31500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:45:54.365461 31500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:45:54.365753 31500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:45:54.365787 31500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.365821 31500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:45:54.365841 31500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:45:54.372228 31500 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:43209
I20260925 14:45:54.372373 31629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:43209 every 8 connection(s)
I20260925 14:45:54.372663 31500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.378753 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31500
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 14:45:54.380752 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.373463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33431 (local address 127.21.205.254:35071)
0925 14:45:54.373535 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.373538 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.373744 (+ 206us) server_negotiation.cc:407] Connection header received
0925 14:45:54.374227 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.374228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.374238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.374321 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.375287 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.375920 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.376935 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.377150 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.377207 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.380691 (+ 3484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:33431: BlockingRecv error: recv got EOF from 127.21.205.195:33431 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:45:54.380859 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.373441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:33431)
0925 14:45:54.373615 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.373659 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.374143 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.374387 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.374390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.374983 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.375191 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.375197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.376002 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.376003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.376856 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.376859 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.377224 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.377225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.377349 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.377870 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.377880 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.380632 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
W20260925 14:45:54.381108 31630 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:35071 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.205.254:35071: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.386600 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.381355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:33723)
0925 14:45:54.381406 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.381417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.381454 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.381669 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.381670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.381815 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.381892 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.381894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.382715 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.382717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.383522 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.383525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.383965 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.383966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.384043 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.384385 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.384391 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.386490 (+ 2099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:45:54.386631 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.381457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33723 (local address 127.21.205.254:35071)
0925 14:45:54.381525 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.381527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.381536 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:54.381575 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.381576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.381584 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.381648 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.381981 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.382640 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.383609 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.383895 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.383960 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.386562 (+ 2602us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:33723: BlockingRecv error: recv got EOF from 127.21.205.195:33723 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:45:54.389256 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.379389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.21.205.254:35071)
0925 14:45:54.379535 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.379537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.379546 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:54.379575 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.379576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.379591 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.379670 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.380505 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.381228 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.383042 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.383232 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.383282 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.385058 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:45:54.385063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:45:54.385063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:45:54.385077 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:45:54.387686 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:54.388114 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:54.388119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:54.388119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:54.388231 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:45:54.388479 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:45:54.388480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:45:54.388480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:45:54.388743 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:45:54.388904 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0925 14:45:54.389030 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 14:45:54.389163 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:54 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:54.391130 31632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:54.386901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:59561)
0925 14:45:54.386954 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:54.386966 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:54.387012 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:54.387140 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:54.387141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:54.387269 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:54.387296 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.387297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.387971 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.387972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.388493 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:54.388494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.388873 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.388874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:54.388929 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.389309 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:54.389314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:54.391076 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:45:54.391232 31364 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:54.386968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:59561 (local address 127.21.205.254:35071)
0925 14:45:54.387005 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:54.387007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:54.387015 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:54.387073 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:54.387074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:54.387083 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:54.387132 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:54.387382 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.387900 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.388587 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:54.388787 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:54.388871 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:54.391149 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:59561: BlockingRecv error: recv got EOF from 127.21.205.195:59561 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 14:45:54.391291 31630 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:55.099040 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:55.094418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:46527)
0925 14:45:55.094570 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:55.094582 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:55.094630 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:55.094778 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:55.094778 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:55.094945 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:55.094998 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.094999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.095640 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.095640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.096254 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.096257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.096604 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.096605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.096653 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.097167 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:55.097172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:55.098965 (+ 1793us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
W20260925 14:45:55.099135 31636 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:55.094481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:46527 (local address 127.21.205.254:35071)
0925 14:45:55.094610 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:55.094614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:55.094627 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:45:55.094689 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:55.094690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:55.094699 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:55.094750 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:55.095062 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.095571 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.096330 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.096533 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.096586 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.099063 (+ 2477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:46527: BlockingRecv error: recv got EOF from 127.21.205.193:46527 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:55.244844 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:55.239831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:57607)
0925 14:45:55.239987 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:55.240000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:55.240098 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:55.240252 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:55.240253 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:55.240437 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:55.240485 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.240487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.241159 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.241160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.241715 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.241717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.242110 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.242111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.242169 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.242657 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:55.242664 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:55.244725 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":66,"threads_started":1}
W20260925 14:45:55.244868 31636 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:55.239913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:57607 (local address 127.21.205.254:35071)
0925 14:45:55.239992 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:55.239994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:55.240081 (+ 87us) server_negotiation.cc:407] Connection header received
0925 14:45:55.240157 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:55.240158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:55.240171 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:55.240247 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:55.240570 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.241070 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.241829 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.242026 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.242110 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.244806 (+ 2696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:57607: BlockingRecv error: recv got EOF from 127.21.205.194:57607 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:55 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:55.396550 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:55.391700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:54261)
0925 14:45:55.391840 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:55.391851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:55.391913 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:55.392056 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:55.392056 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:55.392215 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:55.392268 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.392269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.393000 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.393001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.393613 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:55.393615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.393991 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.393992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:55.394035 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.394507 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:55.394511 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:55.396480 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
W20260925 14:45:55.396652 31636 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:55.391838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:54261 (local address 127.21.205.254:35071)
0925 14:45:55.391905 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:55.391907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:55.391917 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:45:55.391960 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:55.391967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:55.391981 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:55.392041 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:55.392364 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.392885 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.393715 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:55.393916 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:55.393981 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:55.396580 (+ 2599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:54261: BlockingRecv error: recv got EOF from 127.21.205.195:54261 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:56.104208 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:56.099640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:49017)
0925 14:45:56.099872 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:56.099882 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:56.099943 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:56.100071 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:56.100071 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:56.100252 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:56.100302 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.100304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.100931 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.100931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.101394 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.101395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.101779 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.101780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.101841 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.102413 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:56.102416 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:56.104112 (+ 1696us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":129,"threads_started":1}
W20260925 14:45:56.104240 31640 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:56.099758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:49017 (local address 127.21.205.254:35071)
0925 14:45:56.099879 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:56.099881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:56.099941 (+ 60us) server_negotiation.cc:407] Connection header received
0925 14:45:56.099986 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:56.099987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:56.099997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:56.100053 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:56.100372 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.100872 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.101472 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.101692 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.101758 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.104179 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:49017: BlockingRecv error: recv got EOF from 127.21.205.193:49017 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:56.250090 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:56.245453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:36661)
0925 14:45:56.245625 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:56.245636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:56.245718 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:56.245870 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:56.245871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:56.246043 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:56.246093 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.246095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.246745 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.246746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.247294 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.247296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.247659 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.247660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.247727 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.248240 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:56.248246 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:56.249999 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":85,"threads_started":1}
W20260925 14:45:56.250133 31640 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:56.245530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:36661 (local address 127.21.205.254:35071)
0925 14:45:56.245614 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:56.245616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:56.245698 (+ 82us) server_negotiation.cc:407] Connection header received
0925 14:45:56.245779 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:56.245780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:56.245792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:56.245865 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:56.246168 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.246659 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.247374 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.247572 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.247658 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.250073 (+ 2415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:36661: BlockingRecv error: recv got EOF from 127.21.205.194:36661 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:56 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:56.401563 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:56.397068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:42385)
0925 14:45:56.397237 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:56.397248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:56.397314 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:56.397453 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:56.397453 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:56.397610 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:56.397662 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.397664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.398310 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.398310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.398836 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:56.398838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.399160 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.399161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:56.399208 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.399694 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:56.399698 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:56.401505 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
W20260925 14:45:56.401701 31640 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:56.397265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:42385 (local address 127.21.205.254:35071)
0925 14:45:56.397333 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:56.397339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:56.397348 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:45:56.397370 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:56.397370 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:56.397384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:56.397431 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:56.397731 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.398235 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.398908 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:56.399087 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:56.399138 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:56.401608 (+ 2470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:42385: BlockingRecv error: recv got EOF from 127.21.205.195:42385 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:57.109181 31643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:57.104663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:33017)
0925 14:45:57.104804 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:57.104816 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:57.104879 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:57.105034 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:57.105035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:57.105215 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:57.105268 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.105269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.105897 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.105898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.106459 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.106461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.106785 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.106786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.106830 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.107268 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:57.107273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:57.109091 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":87,"threads_started":1}
W20260925 14:45:57.109267 31644 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:57.104715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:33017 (local address 127.21.205.254:35071)
0925 14:45:57.104843 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:57.104845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:57.104857 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:45:57.104941 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:57.104941 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:57.104952 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:57.105016 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:57.105331 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.105832 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.106532 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.106715 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.106771 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.109185 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:33017: BlockingRecv error: recv got EOF from 127.21.205.193:33017 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:57.255643 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:57.250673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:60469)
0925 14:45:57.250792 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:57.250802 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:57.250879 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:57.251107 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:57.251108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:57.251270 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:57.251364 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.251366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.252170 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.252172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.252719 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.252721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.253229 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.253230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.253280 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.253769 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:57.253773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:57.255550 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
W20260925 14:45:57.255755 31644 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:57.250716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:60469 (local address 127.21.205.254:35071)
0925 14:45:57.250770 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:57.250773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:57.250885 (+ 112us) server_negotiation.cc:407] Connection header received
0925 14:45:57.250999 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:57.251001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:57.251015 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:57.251079 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:57.251474 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.252093 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.252804 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.253106 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.253225 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.255666 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:60469: BlockingRecv error: recv got EOF from 127.21.205.194:60469 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:57 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:57.406411 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:57.402148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:38863)
0925 14:45:57.402337 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:57.402348 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:57.402411 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:57.402536 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:57.402536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:57.402693 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:57.402744 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.402745 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.403366 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.403367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.403917 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:57.403919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.404265 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.404266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:57.404305 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.404734 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:57.404738 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:57.406353 (+ 1615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":119,"threads_started":1}
W20260925 14:45:57.406569 31644 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:57.402211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:38863 (local address 127.21.205.254:35071)
0925 14:45:57.402283 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:57.402286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:57.402410 (+ 124us) server_negotiation.cc:407] Connection header received
0925 14:45:57.402455 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:57.402456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:57.402468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:57.402515 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:57.402805 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.403305 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.403999 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:57.404195 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:57.404260 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:57.406471 (+ 2211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:38863: BlockingRecv error: recv got EOF from 127.21.205.195:38863 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:58.114319 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:58.109700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:53717)
0925 14:45:58.109864 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:58.109874 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:58.109930 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:58.110039 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:58.110039 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:58.110212 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:58.110261 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.110263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.110927 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.110928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.111495 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.111497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.111895 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.111897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.111946 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.112396 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:58.112401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:58.114260 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
W20260925 14:45:58.114457 31648 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:58.109768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:53717 (local address 127.21.205.254:35071)
0925 14:45:58.109870 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:58.109873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:58.109934 (+ 61us) server_negotiation.cc:407] Connection header received
0925 14:45:58.109970 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:58.109970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:58.109980 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:58.110035 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:58.110314 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.110829 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.111586 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.111800 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.111856 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.114351 (+ 2495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:53717: BlockingRecv error: recv got EOF from 127.21.205.193:53717 (error 108)
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":45,"threads_started":1}
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:58.260193 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:58.256052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:56001)
0925 14:45:58.256182 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:58.256199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:58.256253 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:58.256371 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:58.256371 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:58.256520 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:58.256571 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.256572 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.257192 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.257192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.257696 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.257698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.258011 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.258012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.258051 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.258453 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:58.258457 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:58.260137 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":84,"threads_started":1}
W20260925 14:45:58.260315 31648 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:58.256101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:56001 (local address 127.21.205.254:35071)
0925 14:45:58.256171 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:58.256173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:58.256259 (+ 86us) server_negotiation.cc:407] Connection header received
0925 14:45:58.256298 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:58.256299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:58.256309 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:58.256360 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:58.256631 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.257127 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.257770 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.257950 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.257999 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.260233 (+ 2234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:56001: BlockingRecv error: recv got EOF from 127.21.205.194:56001 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:58 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:58.411375 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:58.406942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:32989)
0925 14:45:58.407079 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:58.407091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:58.407153 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:58.407292 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:58.407293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:58.407448 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:58.407500 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.407501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.408154 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.408155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.408681 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:58.408682 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.408997 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.408998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:58.409041 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.409549 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:58.409556 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:58.411316 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
W20260925 14:45:58.411474 31648 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:58.407030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:32989 (local address 127.21.205.254:35071)
0925 14:45:58.407097 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:58.407100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:58.407152 (+ 52us) server_negotiation.cc:407] Connection header received
0925 14:45:58.407207 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:58.407207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:58.407219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:58.407274 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:58.407558 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.408069 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.408747 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:58.408924 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:58.408970 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:58.411404 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:32989: BlockingRecv error: recv got EOF from 127.21.205.195:32989 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:59.125185 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:59.114794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:59739)
0925 14:45:59.114962 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:59.114972 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:59.115020 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:59.115214 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:59.115217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:59.115482 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:59.115547 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.115550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.116253 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.116254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.116786 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.116787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.117132 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.117134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.117192 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.123416 (+ 6224us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:59.123421 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:59.125125 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
W20260925 14:45:59.125286 31652 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:59.114866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:59739 (local address 127.21.205.254:35071)
0925 14:45:59.114992 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:59.114994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:59.115005 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:45:59.115092 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:59.115093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:59.115108 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:59.115203 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:59.115631 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.116180 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.116852 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.117041 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.117092 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.125216 (+ 8124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:59739: BlockingRecv error: recv got EOF from 127.21.205.193:59739 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:59.264750 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:59.260637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:51207)
0925 14:45:59.260771 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:59.260780 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:59.260820 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:59.260943 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:59.260943 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:59.261098 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:59.261147 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.261148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.261746 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.261747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.262245 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.262247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.262542 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.262543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.262582 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.262988 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:59.262992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:59.264691 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":56,"threads_started":1}
W20260925 14:45:59.264916 31652 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:59.260748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:51207 (local address 127.21.205.254:35071)
0925 14:45:59.260819 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:59.260820 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:59.260828 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:45:59.260864 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:59.260865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:59.260874 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:59.260922 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:59.261207 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.261677 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.262309 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.262483 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.262530 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.264846 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:51207: BlockingRecv error: recv got EOF from 127.21.205.194:51207 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:45:59 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:45:59.416787 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:45:59.411927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:39141)
0925 14:45:59.412094 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 14:45:59.412105 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:45:59.412173 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:45:59.412323 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:45:59.412324 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:45:59.412495 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:45:59.412549 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.412551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.413270 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.413271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.413805 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:45:59.413807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.414323 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.414324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:45:59.414397 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.414932 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:45:59.414944 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:45:59.416726 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
W20260925 14:45:59.416932 31652 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:45:59.412003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:39141 (local address 127.21.205.254:35071)
0925 14:45:59.412104 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0925 14:45:59.412106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:45:59.412170 (+ 64us) server_negotiation.cc:407] Connection header received
0925 14:45:59.412230 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:45:59.412230 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:45:59.412252 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:45:59.412303 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:45:59.412646 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.413189 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.413912 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:45:59.414219 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:45:59.414305 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:45:59.416844 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:39141: BlockingRecv error: recv got EOF from 127.21.205.195:39141 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:00.130029 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:00.125763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:60835)
0925 14:46:00.125882 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:00.125892 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:00.125946 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:00.126080 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:00.126080 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:00.126239 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:00.126289 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.126290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.126962 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.126963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.127485 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.127486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.127838 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.127839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.127888 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.128306 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:00.128310 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:00.129975 (+ 1665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":72,"threads_started":1}
W20260925 14:46:00.130129 31656 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:00.125832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:60835 (local address 127.21.205.254:35071)
0925 14:46:00.125959 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:00.125961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:00.125974 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:46:00.125995 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:00.125995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:00.126006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:00.126067 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:00.126350 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.126898 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.127553 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.127739 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.127815 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.130060 (+ 2245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:60835: BlockingRecv error: recv got EOF from 127.21.205.193:60835 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":77,"threads_started":1}
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:00.269526 31657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:00.265183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:53299)
0925 14:46:00.265334 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:00.265344 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:00.265401 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:00.265508 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:00.265508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:00.265658 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:00.265732 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.265733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.266348 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.266349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.266924 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.266926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.267249 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.267250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.267293 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.267720 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:00.267724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:00.269468 (+ 1744us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
W20260925 14:46:00.269634 31656 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:00.265225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:53299 (local address 127.21.205.254:35071)
0925 14:46:00.265287 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:00.265290 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:00.265398 (+ 108us) server_negotiation.cc:407] Connection header received
0925 14:46:00.265445 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:00.265446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:00.265457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:00.265503 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:00.265782 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.266257 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.267002 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.267181 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.267231 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.269562 (+ 2331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:53299: BlockingRecv error: recv got EOF from 127.21.205.194:53299 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:00 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:00.421936 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:00.417268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:57021)
0925 14:46:00.417457 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:00.417469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:00.417560 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:00.417690 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:00.417692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:00.417883 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:00.417932 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.417934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.418627 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.418628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.419169 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:00.419171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.419547 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.419547 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:00.419606 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.420063 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:00.420067 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:00.421877 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":109,"threads_started":1}
W20260925 14:46:00.422050 31656 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:00.417361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:57021 (local address 127.21.205.254:35071)
0925 14:46:00.417440 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:00.417442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:00.417543 (+ 101us) server_negotiation.cc:407] Connection header received
0925 14:46:00.417617 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:00.417618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:00.417632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:00.417685 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:00.418011 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.418533 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.419257 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:00.419471 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:00.419547 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:00.421980 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:57021: BlockingRecv error: recv got EOF from 127.21.205.195:57021 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:01.134934 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:01.130476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:37765)
0925 14:46:01.130616 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:01.130626 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:01.130685 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:01.130811 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:01.130812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:01.130982 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:01.131032 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.131034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.131659 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.131660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.132263 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.132265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.132684 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.132685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.132734 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.133237 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:01.133241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:01.134876 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260925 14:46:01.135080 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:01.130562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:37765 (local address 127.21.205.254:35071)
0925 14:46:01.130692 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:01.130694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:01.130711 (+ 17us) server_negotiation.cc:407] Connection header received
0925 14:46:01.130732 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:01.130733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:01.130743 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:01.130796 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:01.131094 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.131599 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.132395 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.132596 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.132659 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.134987 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:37765: BlockingRecv error: recv got EOF from 127.21.205.193:37765 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:01.274425 31661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:01.269978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:41233)
0925 14:46:01.270093 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:01.270105 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:01.270163 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:01.270298 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:01.270298 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:01.270452 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:01.270514 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.270516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.271164 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.271165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.271683 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.271684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.272065 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.272066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.272106 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.272587 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:01.272592 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:01.274367 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
W20260925 14:46:01.274514 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:01.270040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:41233 (local address 127.21.205.254:35071)
0925 14:46:01.270150 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:01.270151 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:01.270160 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:46:01.270220 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:01.270221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:01.270232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:01.270280 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:01.270573 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.271099 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.271751 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.271990 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.272042 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.274447 (+ 2405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:41233: BlockingRecv error: recv got EOF from 127.21.205.194:41233 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:01 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:01.426906 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:01.422516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:33815)
0925 14:46:01.422663 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:01.422672 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:01.422728 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:01.422853 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:01.422854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:01.423006 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:01.423055 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.423057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.423746 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.423747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.424335 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:01.424337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.424642 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.424643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:01.424682 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.425102 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:01.425106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:01.426849 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
W20260925 14:46:01.427027 31660 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:01.422579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33815 (local address 127.21.205.254:35071)
0925 14:46:01.422669 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:01.422671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:01.422716 (+ 45us) server_negotiation.cc:407] Connection header received
0925 14:46:01.422780 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:01.422781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:01.422793 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:01.422834 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:01.423126 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.423677 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.424403 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:01.424580 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:01.424629 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:01.426931 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:33815: BlockingRecv error: recv got EOF from 127.21.205.195:33815 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:02.139943 31663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:02.135399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:50903)
0925 14:46:02.135513 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:02.135524 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:02.135576 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:02.135748 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:02.135748 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:02.135940 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:02.135993 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.135994 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.136609 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.136610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.137140 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.137141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.137475 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.137476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.137516 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.137957 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:02.137961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:02.139837 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
W20260925 14:46:02.140008 31664 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:02.135500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:50903 (local address 127.21.205.254:35071)
0925 14:46:02.135621 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:02.135623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:02.135630 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:46:02.135654 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:02.135655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:02.135670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:02.135725 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:02.136058 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.136547 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.137207 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.137396 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.137463 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.139916 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:50903: BlockingRecv error: recv got EOF from 127.21.205.193:50903 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:02.280114 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:02.275028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:52757)
0925 14:46:02.275236 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:02.275249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:02.275363 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:02.275532 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:02.275533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:02.275731 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:02.275833 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.275835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.276824 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.276825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.277383 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.277385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.277781 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.277782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.277845 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.278283 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:02.278287 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:02.280045 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":106,"threads_started":1}
W20260925 14:46:02.280241 31664 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:02.275123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:52757 (local address 127.21.205.254:35071)
0925 14:46:02.275226 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:02.275228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:02.275347 (+ 119us) server_negotiation.cc:407] Connection header received
0925 14:46:02.275432 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:02.275434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:02.275448 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:02.275528 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:02.275925 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.276737 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.277480 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.277688 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.277780 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.280147 (+ 2367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:52757: BlockingRecv error: recv got EOF from 127.21.205.194:52757 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:02 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:02.431897 31666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:02.427552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:38033)
0925 14:46:02.427684 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:02.427700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:02.427752 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:02.427917 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:02.427918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:02.428074 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:02.428124 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.428126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.428748 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.428749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.429280 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:02.429282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.429576 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.429577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:02.429616 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.430060 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:02.430064 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:02.431773 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":81,"threads_started":1}
W20260925 14:46:02.432005 31664 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:02.427609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:38033 (local address 127.21.205.254:35071)
0925 14:46:02.427671 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:02.427673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:02.427758 (+ 85us) server_negotiation.cc:407] Connection header received
0925 14:46:02.427825 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:02.427826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:02.427841 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:02.427910 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:02.428181 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.428683 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.429344 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:02.429513 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:02.429559 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:02.431914 (+ 2355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:38033: BlockingRecv error: recv got EOF from 127.21.205.195:38033 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:03.144764 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:03.140392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:56445)
0925 14:46:03.140518 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:03.140530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:03.140593 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:03.140735 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:03.140735 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:03.140941 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:03.140990 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.140991 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.141620 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.141621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.142124 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.142125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.142446 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.142447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.142484 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.142944 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:03.142949 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:03.144671 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":66,"threads_started":1}
W20260925 14:46:03.144812 31668 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:03.140481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56445 (local address 127.21.205.254:35071)
0925 14:46:03.140613 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:03.140615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:03.140621 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:46:03.140642 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:03.140643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:03.140652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:03.140707 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:03.141063 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.141555 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.142193 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.142384 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.142435 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.144744 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:56445: BlockingRecv error: recv got EOF from 127.21.205.193:56445 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:03.286465 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:03.280523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:53647)
0925 14:46:03.280697 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:03.280718 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:03.280793 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:03.280969 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:03.280970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:03.281205 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:03.281276 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.281278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.282151 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.282152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.283089 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.283092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.283563 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.283564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.283640 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.284149 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:03.284155 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:03.286407 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":53,"threads_started":1}
W20260925 14:46:03.286593 31668 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:03.280564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:53647 (local address 127.21.205.254:35071)
0925 14:46:03.280613 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:03.280615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:03.280786 (+ 171us) server_negotiation.cc:407] Connection header received
0925 14:46:03.280864 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:03.280865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:03.280889 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:03.280940 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:03.281360 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.282071 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.283190 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.283484 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.283564 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.286512 (+ 2948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:53647: BlockingRecv error: recv got EOF from 127.21.205.194:53647 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:03 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:03.437143 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:03.432320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:60397)
0925 14:46:03.432459 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:03.432473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:03.432536 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:03.432748 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:03.432749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:03.432938 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:03.432989 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.432991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.433682 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.433683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.434227 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:03.434228 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.434606 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.434607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:03.434677 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.435244 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:03.435249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:03.437080 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
W20260925 14:46:03.437212 31668 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:03.432437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:60397 (local address 127.21.205.254:35071)
0925 14:46:03.432507 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:03.432509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:03.432518 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:46:03.432634 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:03.432635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:03.432656 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:03.432737 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:03.433070 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.433587 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.434309 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:03.434518 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:03.434602 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:03.437166 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:60397: BlockingRecv error: recv got EOF from 127.21.205.195:60397 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:04.153203 31671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:04.145303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:48643)
0925 14:46:04.145449 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:04.145459 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:04.145515 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:04.145651 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:04.145652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:04.145800 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:04.145860 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.145862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.146490 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.146490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.147001 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.147002 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.147307 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.147308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.147350 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.151295 (+ 3945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:04.151299 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:04.153144 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
W20260925 14:46:04.153366 31672 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:04.145370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:48643 (local address 127.21.205.254:35071)
0925 14:46:04.145462 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:04.145464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:04.145511 (+ 47us) server_negotiation.cc:407] Connection header received
0925 14:46:04.145567 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:04.145568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:04.145579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:04.145633 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:04.145926 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.146429 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.147064 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.147235 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.147278 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.153269 (+ 5991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:48643: BlockingRecv error: recv got EOF from 127.21.205.193:48643 (error 108)
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":44,"threads_started":1}
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:04.291167 31673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:04.286986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:46737)
0925 14:46:04.287120 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:04.287136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:04.287190 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:04.287322 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:04.287323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:04.287476 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:04.287527 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.287528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.288223 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.288224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.288733 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.288735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.289046 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.289047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.289085 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.289502 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:04.289506 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:04.291111 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":90,"threads_started":1}
W20260925 14:46:04.291265 31672 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:04.287052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:46737 (local address 127.21.205.254:35071)
0925 14:46:04.287114 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:04.287116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:04.287191 (+ 75us) server_negotiation.cc:407] Connection header received
0925 14:46:04.287248 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:04.287248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:04.287258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:04.287307 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:04.287609 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.288151 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.288811 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.288988 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.289035 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.291201 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:46737: BlockingRecv error: recv got EOF from 127.21.205.194:46737 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:04 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:04.442116 31674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:04.437661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:58493)
0925 14:46:04.437823 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:04.437835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:04.437896 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:04.438031 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:04.438032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:04.438191 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:04.438245 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.438247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.438872 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.438873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.439409 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:04.439411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.439729 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.439730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:04.439843 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.440317 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:04.440321 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:04.442059 (+ 1738us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
W20260925 14:46:04.442214 31672 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:04.437802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:58493 (local address 127.21.205.254:35071)
0925 14:46:04.437877 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:04.437879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:04.437887 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:46:04.437952 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:04.437952 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:04.437963 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:04.438010 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:04.438305 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.438799 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.439477 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:04.439654 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:04.439702 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:04.442145 (+ 2443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:58493: BlockingRecv error: recv got EOF from 127.21.205.195:58493 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:05.158238 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:05.153627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:46367)
0925 14:46:05.153757 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:05.153766 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:05.153814 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:05.153978 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:05.153979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:05.154137 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:05.154186 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.154188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.154854 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.154855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.155387 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.155389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.155741 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.155742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.155834 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.156479 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:05.156485 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:05.158178 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
W20260925 14:46:05.158383 31676 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:05.153722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:46367 (local address 127.21.205.254:35071)
0925 14:46:05.153851 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:05.153853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:05.153860 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:46:05.153883 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:05.153883 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:05.153894 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:05.153960 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:05.154257 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.154770 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.155469 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.155666 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.155727 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.158293 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:46367: BlockingRecv error: recv got EOF from 127.21.205.193:46367 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":67,"threads_started":1}
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:05.296453 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:05.291744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:55191)
0925 14:46:05.291921 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:05.291932 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:05.292005 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:05.292168 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:05.292168 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:05.292365 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:05.292409 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.292411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.293106 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.293107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.293645 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.293647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.294022 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.294023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.294090 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.294526 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:05.294530 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:05.296362 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":120,"threads_started":1}
W20260925 14:46:05.296533 31676 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:05.291936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:55191 (local address 127.21.205.254:35071)
0925 14:46:05.292001 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:05.292003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:05.292013 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:46:05.292067 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:05.292067 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:05.292080 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:05.292145 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:05.292487 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.293016 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.293731 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.293950 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.294023 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.296463 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:55191: BlockingRecv error: recv got EOF from 127.21.205.194:55191 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:05 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:05.447791 31678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:05.442686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:36199)
0925 14:46:05.442836 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:05.442846 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:05.442902 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:05.443011 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:05.443011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:05.443189 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:05.443251 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.443252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.444011 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.444012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.444880 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:05.444882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.445204 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.445205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:05.445241 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.445710 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:05.445714 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:05.447684 (+ 1970us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
W20260925 14:46:05.447849 31676 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:05.442752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:36199 (local address 127.21.205.254:35071)
0925 14:46:05.442815 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:05.442817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:05.442903 (+ 86us) server_negotiation.cc:407] Connection header received
0925 14:46:05.442943 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:05.442944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:05.442957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:05.443005 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:05.443304 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.443927 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.444965 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:05.445142 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:05.445194 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:05.447769 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:36199: BlockingRecv error: recv got EOF from 127.21.205.195:36199 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:06.163865 31679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:06.158762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:54767)
0925 14:46:06.158906 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:06.158919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:06.159003 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:06.159148 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:06.159149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:06.159317 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:06.159368 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.159370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.160045 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.160046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.160580 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.160582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.160945 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.160946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.160992 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.161538 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:06.161543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:06.163726 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":58,"threads_started":1}
W20260925 14:46:06.164011 31680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:06.158861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:54767 (local address 127.21.205.254:35071)
0925 14:46:06.159011 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:06.159013 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:06.159025 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:46:06.159054 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:06.159054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:06.159065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:06.159125 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:06.159438 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.159976 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.160658 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.160854 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.160917 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.163896 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:54767: BlockingRecv error: recv got EOF from 127.21.205.193:54767 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:06.301784 31681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:06.296865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:40463)
0925 14:46:06.296998 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:06.297012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:06.297076 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:06.297212 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:06.297214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:06.297398 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:06.297447 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.297449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.298136 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.298137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.298676 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.298678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.299075 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.299076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.299142 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.299586 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:06.299591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:06.301706 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":76,"threads_started":1}
W20260925 14:46:06.301944 31680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:06.296975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:40463 (local address 127.21.205.254:35071)
0925 14:46:06.297053 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:06.297055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:06.297065 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:46:06.297142 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:06.297143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:06.297155 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:06.297208 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:06.297530 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.298047 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.298765 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.298983 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.299076 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.301873 (+ 2797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:40463: BlockingRecv error: recv got EOF from 127.21.205.194:40463 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:06 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:06.453414 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:06.448362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:46391)
0925 14:46:06.448499 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:06.448519 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:06.448595 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:06.448756 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:06.448757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:06.448919 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:06.448981 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.448983 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.449626 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.449626 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.450224 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:06.450226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.450638 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.450639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:06.450751 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.451298 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:06.451304 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:06.453355 (+ 2051us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260925 14:46:06.453593 31680 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:06.448457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:46391 (local address 127.21.205.254:35071)
0925 14:46:06.448569 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:06.448570 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:06.448604 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:46:06.448660 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:06.448661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:06.448678 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:06.448736 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:06.449058 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.449553 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.450294 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:06.450524 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:06.450621 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:06.453515 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:46391: BlockingRecv error: recv got EOF from 127.21.205.195:46391 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:07.169010 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:07.164381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:44571)
0925 14:46:07.164526 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:07.164537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:07.164590 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:07.164799 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:07.164800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:07.164969 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:07.165018 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.165020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.165694 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.165694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.166235 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.166236 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.166584 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.166585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.166626 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.167072 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:07.167077 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:07.168896 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
W20260925 14:46:07.169072 31684 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:07.164497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:44571 (local address 127.21.205.254:35071)
0925 14:46:07.164640 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:07.164643 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:07.164657 (+ 14us) server_negotiation.cc:407] Connection header received
0925 14:46:07.164690 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:07.164691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:07.164701 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:07.164778 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:07.165095 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.165617 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.166316 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.166516 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.166572 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.168991 (+ 2419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:44571: BlockingRecv error: recv got EOF from 127.21.205.193:44571 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:07.307340 31685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:07.302331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:59917)
0925 14:46:07.302530 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:07.302540 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:07.302616 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:07.302775 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:07.302776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:07.302945 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:07.302995 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.302997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.303732 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.303734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.304341 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.304342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.304698 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.304699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.304753 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.305308 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:07.305314 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:07.307236 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":103,"threads_started":1}
W20260925 14:46:07.307431 31684 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:07.302473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59917 (local address 127.21.205.254:35071)
0925 14:46:07.302540 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:07.302542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:07.302621 (+ 79us) server_negotiation.cc:407] Connection header received
0925 14:46:07.302684 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:07.302685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:07.302698 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:07.302744 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:07.303103 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.303627 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.304430 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.304627 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.304687 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.307368 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:59917: BlockingRecv error: recv got EOF from 127.21.205.194:59917 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:07 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:07.458657 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:07.453960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:52997)
0925 14:46:07.454097 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:07.454109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:07.454178 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:07.454373 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:07.454374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:07.454562 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:07.454609 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.454611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.455305 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.455306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.455890 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:07.455893 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.456268 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.456269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:07.456335 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.456788 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:07.456792 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:07.458582 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
W20260925 14:46:07.458875 31684 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:07.454044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:52997 (local address 127.21.205.254:35071)
0925 14:46:07.454101 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:07.454104 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:07.454187 (+ 83us) server_negotiation.cc:407] Connection header received
0925 14:46:07.454265 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:07.454266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:07.454281 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:07.454358 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:07.454696 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.455211 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.455992 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:07.456189 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:07.456269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:07.458813 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:52997: BlockingRecv error: recv got EOF from 127.21.205.195:52997 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:08.174868 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:08.169806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:34361)
0925 14:46:08.169983 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:08.169997 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:08.170078 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:08.170324 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:08.170324 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:08.170491 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:08.170542 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.170543 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.171329 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.171330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.172074 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.172077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.172424 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.172424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.172466 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.172946 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:08.172951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:08.174797 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":65,"threads_started":1}
W20260925 14:46:08.175015 31688 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:08.170032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:34361 (local address 127.21.205.254:35071)
0925 14:46:08.170166 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:08.170168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:08.170177 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:46:08.170212 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:08.170213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:08.170223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:08.170284 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:08.170614 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.171236 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.172145 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.172353 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.172413 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.174922 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:34361: BlockingRecv error: recv got EOF from 127.21.205.193:34361 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:08.312837 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:08.308021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:54451)
0925 14:46:08.308222 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:08.308233 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:08.308328 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:08.308476 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:08.308477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:08.308664 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:08.308713 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.308715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.309428 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.309429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.310028 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.310030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.310422 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.310423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.310483 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.310948 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:08.310953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:08.312726 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":124,"threads_started":1}
W20260925 14:46:08.312904 31688 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:08.308127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:54451 (local address 127.21.205.254:35071)
0925 14:46:08.308205 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:08.308207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:08.308306 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:46:08.308390 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:08.308392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:08.308404 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:08.308461 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:08.308814 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.309336 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.310116 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.310334 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.310421 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.312823 (+ 2402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:54451: BlockingRecv error: recv got EOF from 127.21.205.194:54451 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:08 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.195@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:08.463495 31690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:08.459122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.195:43511)
0925 14:46:08.459264 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:08.459274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:08.459346 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:08.459477 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:08.459478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:08.459643 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:08.459693 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.459695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.460405 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.460406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.460936 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:08.460938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.461263 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.461265 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:08.461312 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.461739 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:08.461743 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:08.463434 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
W20260925 14:46:08.463634 31688 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:08.459184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:43511 (local address 127.21.205.254:35071)
0925 14:46:08.459233 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:08.459236 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:08.459352 (+ 116us) server_negotiation.cc:407] Connection header received
0925 14:46:08.459394 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:08.459394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:08.459406 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:08.459456 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:08.459758 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.460328 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.461015 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:08.461198 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:08.461249 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:08.463552 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.195:43511: BlockingRecv error: recv got EOF from 127.21.205.195:43511 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 25 14:46:09 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:09 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.193@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:09.183998 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.175471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.193:34931)
0925 14:46:09.175627 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:09.175640 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:09.175721 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:09.175901 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:09.175902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:09.176119 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:09.176177 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.176179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.176993 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.176994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.177575 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.177577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.177954 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.177955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.178035 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.181913 (+ 3878us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:09.181919 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:09.183905 (+ 1986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260925 14:46:09.184083 31692 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:09.175590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:34931 (local address 127.21.205.254:35071)
0925 14:46:09.175744 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:09.175746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:09.175757 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:46:09.175809 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:09.175810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:09.175820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:09.175875 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:09.176267 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.176904 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.177656 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.177880 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.177958 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.184008 (+ 6050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.193:34931: BlockingRecv error: recv got EOF from 127.21.205.193:34931 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
Sep 25 14:46:09 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 14:46:09 dist-test-slave-d7lw krb5kdc[30623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.205.194@KRBTEST.COM for oryx/127.21.205.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 14:46:09.318580 31693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.313508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35071 (local address 127.21.205.194:33691)
0925 14:46:09.313751 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:09.313762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:09.313842 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:09.313993 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:09.313993 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:09.314158 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:09.314211 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.314213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.314912 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.314912 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.315461 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.315463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.316018 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.316019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.316070 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.316695 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:09.316708 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:09.318508 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.205.254:35071: Server oryx/127.21.205.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":185,"threads_started":1}
W20260925 14:46:09.318765 31692 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:09.313605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:33691 (local address 127.21.205.254:35071)
0925 14:46:09.313673 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:09.313675 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:09.313844 (+ 169us) server_negotiation.cc:407] Connection header received
0925 14:46:09.313895 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:09.313896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:09.313906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:09.313969 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:09.314317 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.314834 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.315543 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.315755 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.315868 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.318702 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.205.194:33691: BlockingRecv error: recv got EOF from 127.21.205.194:33691 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:46:09.384518 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31211
I20260925 14:46:09.390118 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31365
I20260925 14:46:09.394909 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31500
I20260925 14:46:09.399562 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31141
2026-09-25T14:46:09Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17599 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T14:46:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:46:09Z Disabled control of system clock
I20260925 14:46:09.428373 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35325
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 14:46:09.511530 31697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:09.511817 31697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:09.511842 31697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:09.513116 31697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:46:09.513173 31697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:09.513187 31697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:09.513200 31697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:46:09.513213 31697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:46:09.514655 31697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:09.514897 31697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:09.515170 31697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:09.520568 31702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.520577 31703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:09.520640 31697 server_base.cc:1034] running on GCE node
W20260925 14:46:09.520620 31705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:09.521072 31697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:09.521417 31697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:09.522573 31697 hybrid_clock.cc:648] HybridClock initialized: now 1790347569522554 us; error 42 us; skew 500 ppm
I20260925 14:46:09.523957 31697 webserver.cc:460] Webserver started at http://127.21.205.254:41919/ using document root <none> and password file <none>
I20260925 14:46:09.524220 31697 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:09.524271 31697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:09.524457 31697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:09.525616 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "cfde733ae0a246ffb8aeed8f6d34678e"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "e9d0c46cbb7fdba4129c41e364a8be4c"
server_key_iv: "c4368dd3b94bbd1ed438f32923f91ecb"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.526021 31697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "cfde733ae0a246ffb8aeed8f6d34678e"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "e9d0c46cbb7fdba4129c41e364a8be4c"
server_key_iv: "c4368dd3b94bbd1ed438f32923f91ecb"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.527467 31697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:46:09.528388 31711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.528577 31697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:46:09.528692 31697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "cfde733ae0a246ffb8aeed8f6d34678e"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "e9d0c46cbb7fdba4129c41e364a8be4c"
server_key_iv: "c4368dd3b94bbd1ed438f32923f91ecb"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.528784 31697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:09.549222 31697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:09.549589 31697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:09.549788 31697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:09.554379 31697 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:35325
I20260925 14:46:09.554483 31764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:35325 every 8 connection(s)
I20260925 14:46:09.554852 31697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:46:09.555577 31765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:09.558382 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e: Bootstrap starting.
I20260925 14:46:09.558987 31765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:09.559237 31765 log.cc:824] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:09.560010 31765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e: No bootstrap required, opened a new log
I20260925 14:46:09.561625 31765 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } }
I20260925 14:46:09.561777 31765 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:09.561801 31765 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfde733ae0a246ffb8aeed8f6d34678e, State: Initialized, Role: FOLLOWER
I20260925 14:46:09.561959 31765 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } }
I20260925 14:46:09.562044 31765 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:46:09.562067 31765 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:46:09.562098 31765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:09.562642 31765 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } }
I20260925 14:46:09.562731 31765 leader_election.cc:302] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfde733ae0a246ffb8aeed8f6d34678e; no voters:
I20260925 14:46:09.562970 31765 leader_election.cc:288] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:46:09.562963 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31697
I20260925 14:46:09.563056 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:46:09.563047 31768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:09.563234 31768 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [term 1 LEADER]: Becoming Leader. State: Replica: cfde733ae0a246ffb8aeed8f6d34678e, State: Running, Role: LEADER
I20260925 14:46:09.563258 31765 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:46:09.563280 22327 external_mini_cluster.cc:1428] Setting key c3faee469155f18e38b66bc94e829466
I20260925 14:46:09.563342 31768 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } }
I20260925 14:46:09.563688 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfde733ae0a246ffb8aeed8f6d34678e. Latest consensus state: current_term: 1 leader_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } } }
I20260925 14:46:09.563764 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:09.563899 31770 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfde733ae0a246ffb8aeed8f6d34678e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35325 } } }
I20260925 14:46:09.563974 31770 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:09.564014 31775 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:46:09.564567 31775 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:46:09.566306 31775 catalog_manager.cc:1347] Generated new cluster ID: c93f61e478504f3c9181aed13ae647a9
I20260925 14:46:09.566363 31775 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:46:09.569679 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.564035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.21.205.254:35325)
0925 14:46:09.564453 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:09.564463 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:09.564496 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:46:09.564636 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:09.564646 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:09.564736 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:09.565278 (+ 542us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:09.565402 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.566662 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.567519 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.568034 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.568169 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.568937 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:09.568941 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:09.568942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:09.569125 (+ 183us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:09.569159 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:09.569171 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:09.569248 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:09.569321 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 14:46:09.569509 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":73,"threads_started":1}
I20260925 14:46:09.588631 31775 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:46:09.589202 31775 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:46:09.599498 31775 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfde733ae0a246ffb8aeed8f6d34678e: Generated new TSK 0
I20260925 14:46:09.599736 31775 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:46:09.608323 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--builtin_ntp_servers=127.21.205.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 14:46:09.699608 31789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:09.699868 31789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:09.699893 31789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:09.701174 31789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:09.701223 31789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:09.701251 31789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:46:09.702666 31789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:09.702960 31789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:09.703298 31789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:09.709419 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.709430 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.709497 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:09.709771 31789 server_base.cc:1034] running on GCE node
I20260925 14:46:09.710021 31789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:09.710287 31789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:09.711445 31789 hybrid_clock.cc:648] HybridClock initialized: now 1790347569711438 us; error 37 us; skew 500 ppm
I20260925 14:46:09.712740 31789 webserver.cc:460] Webserver started at http://127.21.205.193:46107/ using document root <none> and password file <none>
I20260925 14:46:09.713021 31789 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:09.713096 31789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:09.713250 31789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:09.714425 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "cf089fcd6220479dbe7f8e04e0dd2721"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "98c1d90c420ebb7ef12d126805c89684"
server_key_iv: "d28ad6ec7c62dc97a0d487fd06ca61ad"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.714810 31789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "cf089fcd6220479dbe7f8e04e0dd2721"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "98c1d90c420ebb7ef12d126805c89684"
server_key_iv: "d28ad6ec7c62dc97a0d487fd06ca61ad"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.716674 31789 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260925 14:46:09.717448 31803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.717610 31789 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:46:09.717716 31789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "cf089fcd6220479dbe7f8e04e0dd2721"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "98c1d90c420ebb7ef12d126805c89684"
server_key_iv: "d28ad6ec7c62dc97a0d487fd06ca61ad"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.717816 31789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:09.729735 31789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:09.730100 31789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:09.730264 31789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:09.730508 31789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:09.730873 31789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:09.730937 31789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.730979 31789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:09.731009 31789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.737036 31789 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:42855
I20260925 14:46:09.737082 31917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:42855 every 8 connection(s)
I20260925 14:46:09.737458 31789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:46:09.742838 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.738343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:58419 (local address 127.21.205.254:35325)
0925 14:46:09.738410 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:09.738412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:09.738640 (+ 228us) server_negotiation.cc:407] Connection header received
0925 14:46:09.739140 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:09.739141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:09.739152 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:09.739211 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:09.739916 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.740420 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.741451 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.741636 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.741692 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.742600 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:09.742601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:09.742601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:09.742606 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:09.742618 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:09.742621 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:09.742661 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:09.742743 (+ 82us) server_negotiation.cc:299] Negotiation successful
0925 14:46:09.742772 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:46:09.743052 31920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.738294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35325 (local address 127.21.205.193:58419)
0925 14:46:09.738501 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:09.738546 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:09.739051 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:09.739281 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:09.739285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:09.739293 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:09.739831 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.739837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.740512 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.740513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.741366 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.741368 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.741715 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.741715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.741836 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.742463 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:09.742464 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:09.742496 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:09.742521 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:09.742675 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:09.742679 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:09.742682 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:09.742737 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:46:09.742830 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260925 14:46:09.743765 31918 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35325
I20260925 14:46:09.743933 31918 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:09.744143 31918 heartbeater.cc:511] Master 127.21.205.254:35325 requested a full tablet report, sending...
I20260925 14:46:09.744395 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31789
I20260925 14:46:09.744486 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:46:09.744705 22327 external_mini_cluster.cc:1428] Setting key b2ebf32668249154db0738422fe2bcae
I20260925 14:46:09.744850 31728 ts_manager.cc:194] Registered new tserver with Master: cf089fcd6220479dbe7f8e04e0dd2721 (127.21.205.193:42855)
I20260925 14:46:09.745890 31728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.193:58419
I20260925 14:46:09.748536 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--builtin_ntp_servers=127.21.205.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 14:46:09.834964 31921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:09.835192 31921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:09.835224 31921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:09.836515 31921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:09.836563 31921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:09.836591 31921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:46:09.838034 31921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:09.838326 31921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:09.838660 31921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:09.844043 31929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.844012 31926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.844147 31927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:09.844342 31921 server_base.cc:1034] running on GCE node
I20260925 14:46:09.844643 31921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:09.844883 31921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:09.846068 31921 hybrid_clock.cc:648] HybridClock initialized: now 1790347569846060 us; error 37 us; skew 500 ppm
I20260925 14:46:09.847424 31921 webserver.cc:460] Webserver started at http://127.21.205.194:40369/ using document root <none> and password file <none>
I20260925 14:46:09.847681 31921 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:09.847756 31921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:09.847944 31921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:09.849107 31921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "99f0d2f5e27d49c7a9e4247318c1be83"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "4e35d4907db0d08fcbf49b3c6c8bf3e6"
server_key_iv: "8a8bbffe887e2fab48db9fd6dcff5eac"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.849529 31921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "99f0d2f5e27d49c7a9e4247318c1be83"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "4e35d4907db0d08fcbf49b3c6c8bf3e6"
server_key_iv: "8a8bbffe887e2fab48db9fd6dcff5eac"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.850909 31921 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:46:09.851671 31935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.851889 31921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:09.852000 31921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "99f0d2f5e27d49c7a9e4247318c1be83"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "4e35d4907db0d08fcbf49b3c6c8bf3e6"
server_key_iv: "8a8bbffe887e2fab48db9fd6dcff5eac"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.852083 31921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:09.861100 31921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:09.861461 31921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:09.861634 31921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:09.861881 31921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:09.862275 31921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:09.862332 31921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.862378 31921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:09.862403 31921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.868772 31921 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:33359
I20260925 14:46:09.868818 32048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:33359 every 8 connection(s)
I20260925 14:46:09.869181 31921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:46:09.874550 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.869971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:55815 (local address 127.21.205.254:35325)
0925 14:46:09.870017 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:09.870019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:09.870289 (+ 270us) server_negotiation.cc:407] Connection header received
0925 14:46:09.870813 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:09.870814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:09.870825 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:09.870877 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:09.871561 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.872117 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.873176 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.873371 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.873423 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.874250 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:09.874251 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:09.874251 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:09.874257 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:09.874281 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:09.874285 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:09.874326 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:09.874452 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 14:46:09.874479 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:46:09.874569 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 31921
I20260925 14:46:09.874670 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:46:09.874853 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:09.869959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35325 (local address 127.21.205.194:55815)
0925 14:46:09.870150 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:09.870187 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:09.870708 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:09.870936 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:09.870939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:09.870948 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:09.871476 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.871480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.872236 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.872237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.873090 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:09.873093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:09.873447 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:09.873448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:09.873565 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:09.874109 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:09.874110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:09.874139 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:09.874163 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:09.874364 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:09.874370 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:09.874373 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:09.874447 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:46:09.874568 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":93,"threads_started":1}
I20260925 14:46:09.874910 22327 external_mini_cluster.cc:1428] Setting key 641ffeba579afaa5e1deb11646a1d9cc
I20260925 14:46:09.875351 32049 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35325
I20260925 14:46:09.875466 32049 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:09.875735 32049 heartbeater.cc:511] Master 127.21.205.254:35325 requested a full tablet report, sending...
I20260925 14:46:09.876045 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--builtin_ntp_servers=127.21.205.212:44109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 14:46:09.876171 31728 ts_manager.cc:194] Registered new tserver with Master: 99f0d2f5e27d49c7a9e4247318c1be83 (127.21.205.194:33359)
I20260925 14:46:09.876552 31728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.194:55815
W20260925 14:46:09.962117 32052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:09.962335 32052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:09.962358 32052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:09.963770 32052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:09.963860 32052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:09.963912 32052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:46:09.965423 32052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:35325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:09.965687 32052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:09.966019 32052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:09.971184 32060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.971184 32057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:09.971215 32058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:09.971477 32052 server_base.cc:1034] running on GCE node
I20260925 14:46:09.971704 32052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:09.972002 32052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:09.973133 32052 hybrid_clock.cc:648] HybridClock initialized: now 1790347569973109 us; error 40 us; skew 500 ppm
I20260925 14:46:09.974421 32052 webserver.cc:460] Webserver started at http://127.21.205.195:46675/ using document root <none> and password file <none>
I20260925 14:46:09.974696 32052 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:09.974773 32052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:09.974936 32052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:09.976141 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "a03eb11e459241558d22d5180156ce57"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "2b3c45ca5121f04fb228965b3fe85e76"
server_key_iv: "41eb1a794dbc860c1fd1bf375e83ba25"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.976513 32052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "a03eb11e459241558d22d5180156ce57"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "2b3c45ca5121f04fb228965b3fe85e76"
server_key_iv: "41eb1a794dbc860c1fd1bf375e83ba25"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.977859 32052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:46:09.978590 32066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.978780 32052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:46:09.978888 32052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "a03eb11e459241558d22d5180156ce57"
format_stamp: "Formatted at 2026-09-25 14:46:09 on dist-test-slave-d7lw"
server_key: "2b3c45ca5121f04fb228965b3fe85e76"
server_key_iv: "41eb1a794dbc860c1fd1bf375e83ba25"
server_key_version: "encryptionkey@0"
I20260925 14:46:09.978967 32052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:09.994311 32052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:09.994673 32052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:09.994843 32052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:09.995103 32052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:09.995455 32052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:09.995509 32052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:09.995551 32052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:09.995581 32052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.001709 32052 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:33687
I20260925 14:46:10.001751 32182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:33687 every 8 connection(s)
I20260925 14:46:10.002177 32052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:46:10.008056 31777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.003087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:49863 (local address 127.21.205.254:35325)
0925 14:46:10.003178 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.003181 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.003413 (+ 232us) server_negotiation.cc:407] Connection header received
0925 14:46:10.004091 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.004091 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.004103 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.004166 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.004935 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:10.005496 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:10.006541 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:10.006727 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:10.006778 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:10.007666 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:10.007668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:10.007668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:10.007674 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:10.007691 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:10.007695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:10.007742 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:10.007953 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 14:46:10.007987 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 14:46:10.008283 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.003053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35325 (local address 127.21.205.195:49863)
0925 14:46:10.003258 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:10.003303 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:10.003971 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:10.004298 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:10.004302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:10.004312 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:10.004844 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:10.004848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:10.005594 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:10.005595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:10.006452 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:10.006454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:10.006807 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:10.006808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:10.006930 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:10.007516 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:10.007516 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:10.007546 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:10.007571 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:10.007845 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:10.007854 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:10.007857 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:10.007946 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 14:46:10.008045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260925 14:46:10.008853 32183 heartbeater.cc:347] Connected to a master server at 127.21.205.254:35325
I20260925 14:46:10.008945 32183 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:10.009112 32183 heartbeater.cc:511] Master 127.21.205.254:35325 requested a full tablet report, sending...
I20260925 14:46:10.009498 31728 ts_manager.cc:194] Registered new tserver with Master: a03eb11e459241558d22d5180156ce57 (127.21.205.195:33687)
I20260925 14:46:10.009845 31728 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.195:49863
I20260925 14:46:10.011557 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32052
I20260925 14:46:10.011636 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:46:10.011863 22327 external_mini_cluster.cc:1428] Setting key 01166fe07b0bda659802bc7115c2745c
I20260925 14:46:10.012827 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 14:46:10.014670 31777 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:10.014222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39574 (local address 127.21.205.254:35325)
0925 14:46:10.014269 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.014271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.014378 (+ 107us) server_negotiation.cc:407] Connection header received
0925 14:46:10.014417 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.014417 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.014429 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.014483 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.014608 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39574: BlockingRecv error: recv got EOF from 127.0.0.1:39574 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:46:10.014775 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31789
I20260925 14:46:10.021108 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31921
I20260925 14:46:10.026562 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32052
I20260925 14:46:10.031141 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 31697
2026-09-25T14:46:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (630 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T14:46:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:46:10Z Disabled control of system clock
I20260925 14:46:10.058626 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33611
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 14:46:10.143548 32194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:10.143868 32194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:10.143923 32194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:10.145148 32194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:46:10.145241 32194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:10.145262 32194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:10.145282 32194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:46:10.145339 32194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:46:10.146770 32194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33611
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:10.147073 32194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:10.147394 32194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:10.152721 32202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.152766 32200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.152792 32199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.152899 32194 server_base.cc:1034] running on GCE node
I20260925 14:46:10.153272 32194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:10.153617 32194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:10.154822 32194 hybrid_clock.cc:648] HybridClock initialized: now 1790347570154815 us; error 36 us; skew 500 ppm
I20260925 14:46:10.156167 32194 webserver.cc:460] Webserver started at http://127.21.205.254:36773/ using document root <none> and password file <none>
I20260925 14:46:10.156424 32194 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:10.156481 32194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:10.156670 32194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:10.157824 32194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "744882dfa7aa45c0bdbbe97619186f39"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "64ff23e7d8aad2b6bbe474e31718d7b4"
server_key_iv: "13ee716d5f43144792b6198814d864b5"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.158217 32194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "744882dfa7aa45c0bdbbe97619186f39"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "64ff23e7d8aad2b6bbe474e31718d7b4"
server_key_iv: "13ee716d5f43144792b6198814d864b5"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.159601 32194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:46:10.160490 32208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.160722 32194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:10.160836 32194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "744882dfa7aa45c0bdbbe97619186f39"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "64ff23e7d8aad2b6bbe474e31718d7b4"
server_key_iv: "13ee716d5f43144792b6198814d864b5"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.160930 32194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:10.164685 32194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:10.164996 32194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:10.165153 32194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:10.169056 32194 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33611
I20260925 14:46:10.169092 32261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33611 every 8 connection(s)
I20260925 14:46:10.169454 32194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:46:10.170202 32262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:10.173306 32262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39: Bootstrap starting.
I20260925 14:46:10.173667 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32194
I20260925 14:46:10.173779 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:46:10.173971 32262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:10.173982 22327 external_mini_cluster.cc:1428] Setting key 4ed509cdf280f89c91ce5ec93d32fd9e
I20260925 14:46:10.174362 32262 log.cc:824] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:10.175169 32262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39: No bootstrap required, opened a new log
I20260925 14:46:10.176765 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.174590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.21.205.254:33611)
0925 14:46:10.174831 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.174841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.174881 (+ 40us) server_negotiation.cc:407] Connection header received
0925 14:46:10.175385 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.175392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.175434 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.175559 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.176290 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:10.176293 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:10.176293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:10.176298 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:10.176339 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:10.176349 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:10.176422 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:10.176497 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 14:46:10.176570 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":100,"threads_started":1}
I20260925 14:46:10.177208 32262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } }
I20260925 14:46:10.177378 32262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:10.177423 32262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 744882dfa7aa45c0bdbbe97619186f39, State: Initialized, Role: FOLLOWER
I20260925 14:46:10.177565 32262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } }
I20260925 14:46:10.177659 32262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:46:10.177704 32262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:46:10.177757 32262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:10.178710 32262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } }
I20260925 14:46:10.178838 32262 leader_election.cc:302] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 744882dfa7aa45c0bdbbe97619186f39; no voters:
I20260925 14:46:10.179042 32262 leader_election.cc:288] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:46:10.179092 32267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:10.179265 32267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [term 1 LEADER]: Becoming Leader. State: Replica: 744882dfa7aa45c0bdbbe97619186f39, State: Running, Role: LEADER
I20260925 14:46:10.179376 32262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:46:10.179354 32267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } }
I20260925 14:46:10.179736 32268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "744882dfa7aa45c0bdbbe97619186f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } } }
I20260925 14:46:10.179924 32268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:10.179749 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 744882dfa7aa45c0bdbbe97619186f39. Latest consensus state: current_term: 1 leader_uuid: "744882dfa7aa45c0bdbbe97619186f39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "744882dfa7aa45c0bdbbe97619186f39" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33611 } } }
I20260925 14:46:10.180223 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:10.180227 32275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:46:10.180819 32275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:46:10.182365 32275 catalog_manager.cc:1347] Generated new cluster ID: 74647aac2a994c52bb268373e5f95324
I20260925 14:46:10.182430 32275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:46:10.188987 32275 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:46:10.189183 32275 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:46:10.199141 32275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 744882dfa7aa45c0bdbbe97619186f39: Generated new TSK 0
I20260925 14:46:10.199378 32275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:46:10.206895 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--builtin_ntp_servers=127.21.205.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 14:46:10.292265 32286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:10.292488 32286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:10.292513 32286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:10.293839 32286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:10.293889 32286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:10.293919 32286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:46:10.295984 32286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:10.296262 32286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:10.296599 32286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:10.303013 32291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.303001 32292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.303270 32286 server_base.cc:1034] running on GCE node
W20260925 14:46:10.303001 32294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.303627 32286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:10.303961 32286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:10.305133 32286 hybrid_clock.cc:648] HybridClock initialized: now 1790347570305115 us; error 32 us; skew 500 ppm
I20260925 14:46:10.306479 32286 webserver.cc:460] Webserver started at http://127.21.205.193:40889/ using document root <none> and password file <none>
I20260925 14:46:10.306768 32286 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:10.306850 32286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:10.307020 32286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:10.308248 32286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "4fdfcab91865492a85538650d273ef62"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "faacdca5f1c1c2182ce26f6555603572"
server_key_iv: "db8e5eefc90c65d24927c3ffabb733de"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.308626 32286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "4fdfcab91865492a85538650d273ef62"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "faacdca5f1c1c2182ce26f6555603572"
server_key_iv: "db8e5eefc90c65d24927c3ffabb733de"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.310040 32286 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260925 14:46:10.310873 32300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.311074 32286 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:10.311172 32286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "4fdfcab91865492a85538650d273ef62"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "faacdca5f1c1c2182ce26f6555603572"
server_key_iv: "db8e5eefc90c65d24927c3ffabb733de"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.311262 32286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:10.314911 32286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:10.315214 32286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:10.315371 32286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:10.315598 32286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:10.316008 32286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:10.316063 32286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.316105 32286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:10.316133 32286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.322177 32286 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:41089
I20260925 14:46:10.322217 32414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:41089 every 8 connection(s)
I20260925 14:46:10.322564 32286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:46:10.325729 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.323298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:35025 (local address 127.21.205.254:33611)
0925 14:46:10.323344 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.323346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.323564 (+ 218us) server_negotiation.cc:407] Connection header received
0925 14:46:10.324207 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.324208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.324216 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.324265 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.325446 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:10.325447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:10.325448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:10.325452 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:10.325476 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:10.325481 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:10.325560 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:10.325641 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 14:46:10.325657 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:46:10.325933 32417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.323253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33611 (local address 127.21.205.193:35025)
0925 14:46:10.323437 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:10.323473 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:10.324083 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:10.324346 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:10.324350 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:10.324360 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:10.325271 (+ 911us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:10.325271 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:10.325315 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:10.325345 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:10.325557 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:10.325564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:10.325567 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:10.325649 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 14:46:10.325715 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
I20260925 14:46:10.326437 32415 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33611
I20260925 14:46:10.326550 32415 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:10.326714 32415 heartbeater.cc:511] Master 127.21.205.254:33611 requested a full tablet report, sending...
I20260925 14:46:10.327260 32225 ts_manager.cc:194] Registered new tserver with Master: 4fdfcab91865492a85538650d273ef62 (127.21.205.193:41089)
I20260925 14:46:10.332036 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32286
I20260925 14:46:10.332146 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:46:10.332353 22327 external_mini_cluster.cc:1428] Setting key d086f68fdbebe83206c8454f7f4a1f58
I20260925 14:46:10.333371 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--builtin_ntp_servers=127.21.205.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 14:46:10.417178 32418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:10.417402 32418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:10.417428 32418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:10.418605 32418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:10.418646 32418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:10.418673 32418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:46:10.420094 32418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:10.420311 32418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:10.420593 32418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:10.425812 32423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.425840 32424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.425954 32418 server_base.cc:1034] running on GCE node
W20260925 14:46:10.425832 32426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.426311 32418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:10.426582 32418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:10.427748 32418 hybrid_clock.cc:648] HybridClock initialized: now 1790347570427733 us; error 44 us; skew 500 ppm
I20260925 14:46:10.429111 32418 webserver.cc:460] Webserver started at http://127.21.205.194:44449/ using document root <none> and password file <none>
I20260925 14:46:10.429344 32418 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:10.429386 32418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:10.429478 32418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:10.430563 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "4b65519521754c059906b5c15f12ef06"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "f0ed071e2fd4795368c1e0d76e6fe3de"
server_key_iv: "eeb70767d7ae1bb24a783bdb01142add"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.430871 32418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "4b65519521754c059906b5c15f12ef06"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "f0ed071e2fd4795368c1e0d76e6fe3de"
server_key_iv: "eeb70767d7ae1bb24a783bdb01142add"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.432220 32418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:46:10.433001 32432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.433202 32418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:10.433291 32418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "4b65519521754c059906b5c15f12ef06"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "f0ed071e2fd4795368c1e0d76e6fe3de"
server_key_iv: "eeb70767d7ae1bb24a783bdb01142add"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.433352 32418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:10.436810 32418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:10.437049 32418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:10.437145 32418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:10.437304 32418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:10.437554 32418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:10.437584 32418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.437604 32418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:10.437618 32418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.443235 32418 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:41217
I20260925 14:46:10.443331 32546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:41217 every 8 connection(s)
I20260925 14:46:10.443662 32418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:46:10.446905 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.444478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:60475 (local address 127.21.205.254:33611)
0925 14:46:10.444542 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.444544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.444801 (+ 257us) server_negotiation.cc:407] Connection header received
0925 14:46:10.445337 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.445338 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.445345 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.445407 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.446628 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:10.446630 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:10.446630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:10.446635 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:10.446651 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:10.446655 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:10.446713 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:10.446815 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 14:46:10.446831 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:46:10.447113 32549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.444470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33611 (local address 127.21.205.194:60475)
0925 14:46:10.444674 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:10.444708 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:10.445219 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:10.445486 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:10.445490 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:10.445498 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:10.446468 (+ 970us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:10.446468 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:10.446497 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:10.446528 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:10.446718 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:10.446723 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:10.446726 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:10.446824 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 14:46:10.446887 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260925 14:46:10.447598 32547 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33611
I20260925 14:46:10.447698 32547 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:10.447911 32547 heartbeater.cc:511] Master 127.21.205.254:33611 requested a full tablet report, sending...
I20260925 14:46:10.448329 32225 ts_manager.cc:194] Registered new tserver with Master: 4b65519521754c059906b5c15f12ef06 (127.21.205.194:41217)
I20260925 14:46:10.448360 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32418
I20260925 14:46:10.448482 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:46:10.448709 22327 external_mini_cluster.cc:1428] Setting key dac72d3405fe537942ebcafd4445c9f4
I20260925 14:46:10.449635 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--builtin_ntp_servers=127.21.205.212:44671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 14:46:10.535383 32550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:10.535671 32550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:10.535714 32550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:10.537233 32550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:10.537308 32550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:10.537356 32550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:46:10.538842 32550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:33611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:10.539153 32550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:10.539479 32550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:10.544612 32556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.544591 32555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:10.544602 32558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:10.545023 32550 server_base.cc:1034] running on GCE node
I20260925 14:46:10.545260 32550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:10.545472 32550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:10.546653 32550 hybrid_clock.cc:648] HybridClock initialized: now 1790347570546633 us; error 31 us; skew 500 ppm
I20260925 14:46:10.548044 32550 webserver.cc:460] Webserver started at http://127.21.205.195:44287/ using document root <none> and password file <none>
I20260925 14:46:10.548314 32550 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:10.548388 32550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:10.548538 32550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:10.549714 32550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "5ae82b3d737f42a2a060e4b68f31b6a9"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "7a56d62597007a08005de59f13df4dce"
server_key_iv: "c0d6e46d1b51dcf1f0cb65a3f7249b3d"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.550105 32550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "5ae82b3d737f42a2a060e4b68f31b6a9"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "7a56d62597007a08005de59f13df4dce"
server_key_iv: "c0d6e46d1b51dcf1f0cb65a3f7249b3d"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.551532 32550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:46:10.552367 32564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.552541 32550 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.552645 32550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "5ae82b3d737f42a2a060e4b68f31b6a9"
format_stamp: "Formatted at 2026-09-25 14:46:10 on dist-test-slave-d7lw"
server_key: "7a56d62597007a08005de59f13df4dce"
server_key_iv: "c0d6e46d1b51dcf1f0cb65a3f7249b3d"
server_key_version: "encryptionkey@0"
I20260925 14:46:10.552734 32550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:10.556331 32550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:10.556648 32550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:10.556795 32550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:10.557018 32550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:10.557349 32550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:10.557402 32550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.557443 32550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:10.557471 32550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:10.563401 32550 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:42375
I20260925 14:46:10.563441 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:42375 every 8 connection(s)
I20260925 14:46:10.563859 32550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:46:10.564826 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32550
I20260925 14:46:10.564915 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:46:10.565147 22327 external_mini_cluster.cc:1428] Setting key 507cfc0fbd2a50222a77cfb539f567e4
I20260925 14:46:10.567227 32266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.564645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:58543 (local address 127.21.205.254:33611)
0925 14:46:10.564702 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.564705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.565038 (+ 333us) server_negotiation.cc:407] Connection header received
0925 14:46:10.565775 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.565776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.565783 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.565842 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.567030 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:10.567031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:10.567031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:46:10.567035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:10.567049 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:46:10.567053 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:10.567099 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:10.567160 (+ 61us) server_negotiation.cc:299] Negotiation successful
0925 14:46:10.567172 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:46:10.567418 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:10.564646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:33611 (local address 127.21.205.195:58543)
0925 14:46:10.564881 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:10.564934 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:10.565641 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:10.565925 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:10.565929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:10.565941 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:10.566882 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:46:10.566883 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:10.566914 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:46:10.566943 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:10.567097 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:10.567100 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:10.567103 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:10.567156 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 14:46:10.567208 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260925 14:46:10.567939 32679 heartbeater.cc:347] Connected to a master server at 127.21.205.254:33611
I20260925 14:46:10.568040 32679 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:10.568214 32679 heartbeater.cc:511] Master 127.21.205.254:33611 requested a full tablet report, sending...
I20260925 14:46:10.568594 32225 ts_manager.cc:194] Registered new tserver with Master: 5ae82b3d737f42a2a060e4b68f31b6a9 (127.21.205.195:42375)
I20260925 14:46:10.576972 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 14:46:10.578867 32266 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:46:10.578391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.21.205.254:33611)
0925 14:46:10.578472 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:10.578474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:10.578511 (+ 37us) server_negotiation.cc:407] Connection header received
0925 14:46:10.578575 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:10.578576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:10.578580 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:10.578626 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:10.578775 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53644: BlockingRecv error: recv got EOF from 127.0.0.1:53644 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 14:46:10.578938 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32286
I20260925 14:46:10.583868 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32418
I20260925 14:46:10.588747 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32550
I20260925 14:46:10.593339 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32194
2026-09-25T14:46:10Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (561 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:46:10 dist-test-slave-d7lw krb5kdc[32689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:46:10 dist-test-slave-d7lw krb5kdc[32689](info): set up 2 sockets
Sep 25 14:46:10 dist-test-slave-d7lw krb5kdc[32689](info): commencing operation
krb5kdc: starting...
W20260925 14:46:11.635717 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:46:11 dist-test-slave-d7lw krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:46:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:46:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:46:11.757088 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37105
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:37105
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:11.841535 32702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:11.841838 32702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:11.841894 32702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:11.843219 32702 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:46:11.843308 32702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:11.843331 32702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:11.843353 32702 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:46:11.843385 32702 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:46:11.845155 32702 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:37105
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:37105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:11.845492 32702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:11.845754 32702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:11.850966 32707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:11.851008 32708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:11.850991 32710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:11.851104 32702 server_base.cc:1034] running on GCE node
I20260925 14:46:11.851476 32702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:11.851928 32702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:11.853112 32702 hybrid_clock.cc:648] HybridClock initialized: now 1790347571853097 us; error 45 us; skew 500 ppm
Sep 25 14:46:11 dist-test-slave-d7lw krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347571, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:11.855731 32702 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:46:11.856237 32702 webserver.cc:460] Webserver started at http://127.21.205.254:40673/ using document root <none> and password file <none>
I20260925 14:46:11.856510 32702 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:11.856583 32702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:11.856740 32702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:11.857962 32702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "94efae300ee74985acba380f1a3e1199"
format_stamp: "Formatted at 2026-09-25 14:46:11 on dist-test-slave-d7lw"
server_key: "44223555b2ab4b90bcc5b707b0f639ad"
server_key_iv: "a0053dee108b881bd5ae4a926391b584"
server_key_version: "encryptionkey@0"
I20260925 14:46:11.858357 32702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "94efae300ee74985acba380f1a3e1199"
format_stamp: "Formatted at 2026-09-25 14:46:11 on dist-test-slave-d7lw"
server_key: "44223555b2ab4b90bcc5b707b0f639ad"
server_key_iv: "a0053dee108b881bd5ae4a926391b584"
server_key_version: "encryptionkey@0"
I20260925 14:46:11.859755 32702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:46:11.860664 32717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:11.860852 32702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:11.860970 32702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "94efae300ee74985acba380f1a3e1199"
format_stamp: "Formatted at 2026-09-25 14:46:11 on dist-test-slave-d7lw"
server_key: "44223555b2ab4b90bcc5b707b0f639ad"
server_key_iv: "a0053dee108b881bd5ae4a926391b584"
server_key_version: "encryptionkey@0"
I20260925 14:46:11.861074 32702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:11.873338 32702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:11.875367 32702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:11.875545 32702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:11.879546 32702 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:37105
I20260925 14:46:11.879596 302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:37105 every 8 connection(s)
I20260925 14:46:11.880088 32702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:46:11.881150 303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:11.882758 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 32702
I20260925 14:46:11.882872 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:46:11.883049 22327 external_mini_cluster.cc:1428] Setting key 6e081f7f988161ba96ef9d2d9adc1387
I20260925 14:46:11.884083 303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: Bootstrap starting.
I20260925 14:46:11.884826 303 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:11.885145 303 log.cc:824] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:11.885963 303 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: No bootstrap required, opened a new log
I20260925 14:46:11.888386 303 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } }
I20260925 14:46:11.888597 303 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:11.888646 303 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94efae300ee74985acba380f1a3e1199, State: Initialized, Role: FOLLOWER
I20260925 14:46:11.888824 303 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } }
I20260925 14:46:11.888943 303 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:46:11.888983 303 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:46:11.889041 303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:11.889945 303 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } }
I20260925 14:46:11.890098 303 leader_election.cc:302] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94efae300ee74985acba380f1a3e1199; no voters:
I20260925 14:46:11.890307 303 leader_election.cc:288] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:46:11.890367 308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:11.890533 308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [term 1 LEADER]: Becoming Leader. State: Replica: 94efae300ee74985acba380f1a3e1199, State: Running, Role: LEADER
Sep 25 14:46:11 dist-test-slave-d7lw krb5kdc[32689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:11.890626 303 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:46:11.890636 308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } }
I20260925 14:46:11.891073 309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94efae300ee74985acba380f1a3e1199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } } }
I20260925 14:46:11.891077 310 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94efae300ee74985acba380f1a3e1199. Latest consensus state: current_term: 1 leader_uuid: "94efae300ee74985acba380f1a3e1199" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94efae300ee74985acba380f1a3e1199" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 37105 } } }
I20260925 14:46:11.891183 309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:11.891187 310 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:11.891553 315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:46:11.892352 315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:46:11.894114 315 catalog_manager.cc:1347] Generated new cluster ID: 988bdc98cb224a619e30af4de351694c
I20260925 14:46:11.894171 315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:46:11.895919 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:11.883661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.21.205.254:37105)
0925 14:46:11.883880 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:11.883890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:11.883927 (+ 37us) server_negotiation.cc:407] Connection header received
0925 14:46:11.884497 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:11.884507 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:11.884586 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:11.884740 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:11.885422 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:11.886549 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:11.887615 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:11.888098 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:11.888193 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:11.891512 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:11.891517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:11.891517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:11.891531 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:11.893743 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:11.894239 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:11.894242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:11.894242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:11.894363 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:11.894563 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:11.894564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:11.894565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:11.894879 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:11.894996 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:11.895510 (+ 514us) server_negotiation.cc:299] Negotiation successful
0925 14:46:11.895660 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260925 14:46:11.906683 315 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:46:11.907260 315 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:46:11.919380 315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: Generated new TSK 0
I20260925 14:46:11.919620 315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:46:11.947681 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:12.033386 329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:12.033617 329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:12.033643 329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:12.034911 329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:12.034964 329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:12.034994 329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:46:12.036715 329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:12.037007 329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:12.037263 329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:12.042660 335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.042754 336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.042660 338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:12.042918 329 server_base.cc:1034] running on GCE node
I20260925 14:46:12.043254 329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:12.043529 329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:12.044703 329 hybrid_clock.cc:648] HybridClock initialized: now 1790347572044695 us; error 37 us; skew 500 ppm
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:12.047421 329 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:46:12.047972 329 webserver.cc:460] Webserver started at http://127.21.205.193:38157/ using document root <none> and password file <none>
I20260925 14:46:12.048199 329 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:12.048243 329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:12.048347 329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:12.049418 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "7f48c1a4451f4b5a86635009f80f854b"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "ad0206c454d5a7f0cd0829d9570dc46c"
server_key_iv: "283a18529df9892042def6f2c70e7fdb"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.049742 329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "7f48c1a4451f4b5a86635009f80f854b"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "ad0206c454d5a7f0cd0829d9570dc46c"
server_key_iv: "283a18529df9892042def6f2c70e7fdb"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.051074 329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:46:12.051885 345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.052073 329 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:46:12.052178 329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "7f48c1a4451f4b5a86635009f80f854b"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "ad0206c454d5a7f0cd0829d9570dc46c"
server_key_iv: "283a18529df9892042def6f2c70e7fdb"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.052268 329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:12.061429 329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:12.064196 329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:12.064379 329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:12.064630 329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:12.065018 329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:12.065073 329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.065119 329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:12.065145 329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.072259 329 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:43669
I20260925 14:46:12.072319 462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:43669 every 8 connection(s)
I20260925 14:46:12.072690 329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:46:12.073117 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 329
I20260925 14:46:12.073223 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:46:12.073453 22327 external_mini_cluster.cc:1428] Setting key 87282cee7eff8ddae72203f37d27ee46
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
I20260925 14:46:12.083391 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.073919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:34163 (local address 127.21.205.254:37105)
0925 14:46:12.073959 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.073962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.073970 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:46:12.074600 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.074602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.074615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.074688 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:12.075742 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.076309 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.077391 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.077615 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.077710 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.080268 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:12.080270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:12.080270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:12.080278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:12.081810 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.082285 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.082287 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.082287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.082349 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.082629 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.082630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.082630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.082758 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:12.082837 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.083190 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.083288 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:46:12.083611 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.073649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37105 (local address 127.21.205.193:34163)
0925 14:46:12.073836 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.073886 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.074486 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.074746 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.074749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.075391 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:12.075620 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.075627 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.076416 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.076417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.077291 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.077293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.077727 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.077728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.077869 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.078465 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:12.078474 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:12.080106 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:12.081922 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.081932 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.081936 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.082189 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.082424 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.082425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.082425 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.082548 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.082828 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:12.082835 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:12.083010 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.083166 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.083350 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
I20260925 14:46:12.084295 463 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37105
I20260925 14:46:12.084416 463 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:12.084667 463 heartbeater.cc:511] Master 127.21.205.254:37105 requested a full tablet report, sending...
I20260925 14:46:12.086043 32735 ts_manager.cc:194] Registered new tserver with Master: 7f48c1a4451f4b5a86635009f80f854b (127.21.205.193:43669)
I20260925 14:46:12.086687 32735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:34163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:46:12.099205 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:12.183522 468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:12.183833 468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:12.183899 468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:12.185141 468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:12.185195 468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:12.185223 468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:46:12.186775 468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:12.187031 468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:12.187273 468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:12.192721 474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.192795 473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.192794 476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:12.193136 468 server_base.cc:1034] running on GCE node
I20260925 14:46:12.193397 468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:12.193681 468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:12.194830 468 hybrid_clock.cc:648] HybridClock initialized: now 1790347572194793 us; error 51 us; skew 500 ppm
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:12.197571 468 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:46:12.198101 468 webserver.cc:460] Webserver started at http://127.21.205.194:39133/ using document root <none> and password file <none>
I20260925 14:46:12.198351 468 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:12.198426 468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:12.198580 468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:12.199708 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "07e5e073f6734ed7af8211fdbfb3baac"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "8a92f96070db891848dd4d8ccf88fff3"
server_key_iv: "1336a4500a30cbbce427ab3ae1e45345"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.200184 468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "07e5e073f6734ed7af8211fdbfb3baac"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "8a92f96070db891848dd4d8ccf88fff3"
server_key_iv: "1336a4500a30cbbce427ab3ae1e45345"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.201601 468 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:12.202371 483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.202550 468 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:46:12.202662 468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "07e5e073f6734ed7af8211fdbfb3baac"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "8a92f96070db891848dd4d8ccf88fff3"
server_key_iv: "1336a4500a30cbbce427ab3ae1e45345"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.202745 468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:12.215101 468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:12.217751 468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:12.217936 468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:12.218202 468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:12.218559 468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:12.218614 468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.218658 468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:12.218703 468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.224843 468 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:44547
I20260925 14:46:12.224886 598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:44547 every 8 connection(s)
I20260925 14:46:12.225250 468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:12.234651 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 468
I20260925 14:46:12.234777 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:46:12.234993 22327 external_mini_cluster.cc:1428] Setting key a0b8d34a5af1a33262f767a6e5a2d5d9
I20260925 14:46:12.235477 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.226029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:41967 (local address 127.21.205.254:37105)
0925 14:46:12.226107 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.226109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.226350 (+ 241us) server_negotiation.cc:407] Connection header received
0925 14:46:12.226911 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.226913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.226926 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.226985 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:12.228030 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.228559 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.229633 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.229860 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.229957 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.232482 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:12.232484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:12.232484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:12.232493 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:12.233889 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.234365 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.234366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.234366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.234427 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.234718 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.234719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.234720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.234885 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:12.234950 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.235287 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.235377 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 14:46:12.235710 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.226029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37105 (local address 127.21.205.194:41967)
0925 14:46:12.226211 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.226250 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.226794 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.227070 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.227074 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.227677 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:12.227920 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.227925 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.228668 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.228669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.229514 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.229516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.229969 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.229970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.230108 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.230692 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:12.230702 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:12.232366 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:12.234020 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.234030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.234032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.234277 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.234525 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.234525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.234526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.234646 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.234940 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:12.234948 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:12.235135 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.235265 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.235454 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260925 14:46:12.236202 599 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37105
I20260925 14:46:12.236339 599 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:12.236533 599 heartbeater.cc:511] Master 127.21.205.254:37105 requested a full tablet report, sending...
I20260925 14:46:12.236938 32735 ts_manager.cc:194] Registered new tserver with Master: 07e5e073f6734ed7af8211fdbfb3baac (127.21.205.194:44547)
I20260925 14:46:12.237495 32735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:41967
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:46:12.260852 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:43421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:12.344851 604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:12.345078 604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:12.345103 604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:12.346215 604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:12.346256 604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:12.346283 604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:46:12.347695 604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:43421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:37105
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:12.347999 604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:12.348214 604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:46:12.353580 604 server_base.cc:1034] running on GCE node
W20260925 14:46:12.353603 610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.353608 612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:12.353607 609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:12.354053 604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:12.354339 604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:12.355535 604 hybrid_clock.cc:648] HybridClock initialized: now 1790347572355533 us; error 39 us; skew 500 ppm
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:12.358100 604 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:46:12.358634 604 webserver.cc:460] Webserver started at http://127.21.205.195:34827/ using document root <none> and password file <none>
I20260925 14:46:12.358899 604 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:12.358970 604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:12.359133 604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:46:12.360324 604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "b7a25f67076246d69f9057fb6e883ce7"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "1c9a5a23e662de6c15dca5e5c50ee6e3"
server_key_iv: "5dcffcf9f34d63c1db9ffd824dd4a742"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.360714 604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "b7a25f67076246d69f9057fb6e883ce7"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "1c9a5a23e662de6c15dca5e5c50ee6e3"
server_key_iv: "5dcffcf9f34d63c1db9ffd824dd4a742"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.362176 604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:46:12.363003 619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.363204 604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:12.363310 604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "b7a25f67076246d69f9057fb6e883ce7"
format_stamp: "Formatted at 2026-09-25 14:46:12 on dist-test-slave-d7lw"
server_key: "1c9a5a23e662de6c15dca5e5c50ee6e3"
server_key_iv: "5dcffcf9f34d63c1db9ffd824dd4a742"
server_key_version: "encryptionkey@0"
I20260925 14:46:12.363402 604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:12.382161 604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:12.384371 604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:12.384542 604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:12.384801 604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:12.385164 604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:12.385227 604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.385273 604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:12.385298 604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:12.391944 604 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:33407
I20260925 14:46:12.391984 734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:33407 every 8 connection(s)
I20260925 14:46:12.392400 604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:46:12.396528 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 604
I20260925 14:46:12.396633 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:46:12.396839 22327 external_mini_cluster.cc:1428] Setting key 36b07009cc48f4463ff68fcfef24ccc9
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[32689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:12.402263 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.393135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:43403 (local address 127.21.205.254:37105)
0925 14:46:12.393219 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.393222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.393419 (+ 197us) server_negotiation.cc:407] Connection header received
0925 14:46:12.393942 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.393944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.393955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.394012 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:12.394987 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.395502 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.396851 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.397167 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.397248 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.399567 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:12.399569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:12.399569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:12.399577 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:12.400949 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.401358 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.401358 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.401359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.401410 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.401629 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.401630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.401630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.401740 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:12.401791 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.402087 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.402182 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 14:46:12.402395 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.393108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:37105 (local address 127.21.205.195:43403)
0925 14:46:12.393307 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.393348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.393843 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.394071 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.394075 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.394674 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:12.394888 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.394892 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.395606 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.395607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.396760 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.396764 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.397272 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.397274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.397411 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.397996 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:12.398005 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:12.399456 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:12.401034 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.401041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.401043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.401286 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.401471 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:12.401471 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:12.401472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:12.401575 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:12.401802 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:12.401807 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:12.401937 (+ 130us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.402026 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.402162 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
I20260925 14:46:12.402971 735 heartbeater.cc:347] Connected to a master server at 127.21.205.254:37105
I20260925 14:46:12.403090 735 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:12.403302 735 heartbeater.cc:511] Master 127.21.205.254:37105 requested a full tablet report, sending...
I20260925 14:46:12.403724 32735 ts_manager.cc:194] Registered new tserver with Master: b7a25f67076246d69f9057fb6e883ce7 (127.21.205.195:33407)
I20260925 14:46:12.404156 32735 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:43403
I20260925 14:46:12.408560 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:46:12.415992 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.409875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42552 (local address 127.21.205.254:37105)
0925 14:46:12.409943 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.409945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.410019 (+ 74us) server_negotiation.cc:407] Connection header received
0925 14:46:12.410058 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.410059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.410067 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.410130 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:12.410675 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.411192 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.411967 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.412183 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.412241 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.413561 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:12.413562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:12.413562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:12.413569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:12.414852 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.415177 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.415177 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.415178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.415226 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:12.415403 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:12.415404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:12.415404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:12.415552 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:12.415627 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.415728 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.415856 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:46:12.417523 32735 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:46:12.419215 32735 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:46:12.426266 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.422104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.21.205.195:33407)
0925 14:46:12.422297 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.422302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.422310 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:46:12.422375 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.422377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.422393 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.422638 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.422731 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.423905 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425155 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.425843 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425959 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.426026 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.426158 (+ 132us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.426194 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
I20260925 14:46:12.426297 746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.421769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:33407 (local address 127.0.0.1:53724)
0925 14:46:12.422037 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.422048 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.422099 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.422496 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.422497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.422500 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.422558 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.422560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.424015 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.424016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.425067 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.425069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.425957 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.426047 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.426064 (+ 17us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.426139 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.426196 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":49,"threads_started":1}
I20260925 14:46:12.426725 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.421744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43669 (local address 127.0.0.1:44366)
0925 14:46:12.421922 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.421972 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.422040 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.422472 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.422476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.422479 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.422551 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.422553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.423584 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.423586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.424958 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.424961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.426496 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.426497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.426563 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.426566 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.426616 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.426660 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":69,"threads_started":1}
I20260925 14:46:12.427034 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.421852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44366 (local address 127.21.205.193:43669)
0925 14:46:12.422295 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.422299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.422308 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:46:12.422346 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.422347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.422361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.422635 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.422718 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.423476 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425040 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.425851 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425959 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.426026 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.426902 (+ 876us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.426954 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":90,"threads_started":1}
I20260925 14:46:12.427134 668 tablet_service.cc:1467] Processing CreateTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 (DEFAULT_TABLE table=test-table [id=8b1ca89dc65647f2b404c615bb7f84ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:12.427522 668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd79aac34f094eb8b1de11280a9fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:12.427788 385 tablet_service.cc:1467] Processing CreateTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 (DEFAULT_TABLE table=test-table [id=8b1ca89dc65647f2b404c615bb7f84ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:12.428135 385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd79aac34f094eb8b1de11280a9fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:12.431267 752 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Bootstrap starting.
I20260925 14:46:12.431627 753 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: Bootstrap starting.
I20260925 14:46:12.431823 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.422082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.21.205.194:44547)
0925 14:46:12.422234 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.422239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.422252 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:46:12.422295 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.422298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.422316 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.422402 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.422683 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.423905 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425915 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.431327 (+ 5412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.431420 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.431492 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.431627 (+ 135us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.431692 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260925 14:46:12.432117 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.421956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:44547 (local address 127.0.0.1:48882)
0925 14:46:12.422177 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.422184 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.422218 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.422552 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.422553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.422554 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.422601 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.422602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.425151 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.425151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.425822 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.425824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.431417 (+ 5593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.431419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.431512 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.431518 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.431962 (+ 444us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.432040 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":71,"threads_started":1}
I20260925 14:46:12.432128 752 tablet_bootstrap.cc:654] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:12.432343 753 tablet_bootstrap.cc:654] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:12.432880 753 log.cc:824] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:12.432859 532 tablet_service.cc:1467] Processing CreateTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 (DEFAULT_TABLE table=test-table [id=8b1ca89dc65647f2b404c615bb7f84ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:12.433259 532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd79aac34f094eb8b1de11280a9fbae0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:12.433661 753 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: No bootstrap required, opened a new log
I20260925 14:46:12.433748 753 ts_tablet_manager.cc:1397] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:12.435184 753 raft_consensus.cc:348] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.435362 753 raft_consensus.cc:374] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:12.435410 753 raft_consensus.cc:729] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f48c1a4451f4b5a86635009f80f854b, State: Initialized, Role: FOLLOWER
I20260925 14:46:12.435572 753 consensus_queue.cc:260] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.435931 463 heartbeater.cc:503] Master 127.21.205.254:37105 was elected leader, sending a full tablet report...
I20260925 14:46:12.435941 753 ts_tablet_manager.cc:1428] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:12.436579 752 log.cc:824] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:12.436956 756 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Bootstrap starting.
I20260925 14:46:12.437450 752 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: No bootstrap required, opened a new log
I20260925 14:46:12.437541 752 ts_tablet_manager.cc:1397] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260925 14:46:12.439003 752 raft_consensus.cc:348] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.439141 752 raft_consensus.cc:374] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:12.439163 752 raft_consensus.cc:729] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a25f67076246d69f9057fb6e883ce7, State: Initialized, Role: FOLLOWER
I20260925 14:46:12.439284 752 consensus_queue.cc:260] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.439528 752 ts_tablet_manager.cc:1428] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:12.439574 735 heartbeater.cc:503] Master 127.21.205.254:37105 was elected leader, sending a full tablet report...
I20260925 14:46:12.440219 756 tablet_bootstrap.cc:654] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:12.440798 756 log.cc:824] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:12.441545 756 tablet_bootstrap.cc:492] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: No bootstrap required, opened a new log
I20260925 14:46:12.441532 758 raft_consensus.cc:482] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:46:12.441632 756 ts_tablet_manager.cc:1397] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 14:46:12.441620 758 raft_consensus.cc:504] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.441967 758 leader_election.cc:288] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f48c1a4451f4b5a86635009f80f854b (127.21.205.193:43669), 07e5e073f6734ed7af8211fdbfb3baac (127.21.205.194:44547)
I20260925 14:46:12.443583 756 raft_consensus.cc:348] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.443758 756 raft_consensus.cc:374] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:12.443846 756 raft_consensus.cc:729] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e5e073f6734ed7af8211fdbfb3baac, State: Initialized, Role: FOLLOWER
I20260925 14:46:12.443979 756 consensus_queue.cc:260] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.444278 756 ts_tablet_manager.cc:1428] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:46:12.444303 599 heartbeater.cc:503] Master 127.21.205.254:37105 was elected leader, sending a full tablet report...
I20260925 14:46:12.445457 737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.442143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43669 (local address 127.21.205.195:50351)
0925 14:46:12.442206 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.442219 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.442281 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.442470 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.442471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.442473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.442538 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.442539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.443345 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.443345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.444389 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.444393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.445199 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.445200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.445284 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.445288 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.445342 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.445384 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 14:46:12.445492 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.442226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:50351 (local address 127.21.205.193:43669)
0925 14:46:12.442268 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.442271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.442279 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:46:12.442338 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.442339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.442350 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.442403 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.442616 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.443252 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.444471 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.445100 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.445203 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.445213 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.445400 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.445442 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:46:12.445833 416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "b7a25f67076246d69f9057fb6e883ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f48c1a4451f4b5a86635009f80f854b" is_pre_election: true
I20260925 14:46:12.446002 416 raft_consensus.cc:2454] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7a25f67076246d69f9057fb6e883ce7 in term 0.
I20260925 14:46:12.446219 761 raft_consensus.cc:482] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:46:12.446240 620 leader_election.cc:302] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f48c1a4451f4b5a86635009f80f854b, b7a25f67076246d69f9057fb6e883ce7; no voters:
I20260925 14:46:12.446358 758 raft_consensus.cc:2781] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:46:12.446328 761 raft_consensus.cc:504] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.446400 758 raft_consensus.cc:482] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:46:12.446429 758 raft_consensus.cc:3037] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:12.447214 761 leader_election.cc:288] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f48c1a4451f4b5a86635009f80f854b (127.21.205.193:43669), b7a25f67076246d69f9057fb6e883ce7 (127.21.205.195:33407)
I20260925 14:46:12.447382 758 raft_consensus.cc:504] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.447532 758 leader_election.cc:288] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [CANDIDATE]: Term 1 election: Requested vote from peers 7f48c1a4451f4b5a86635009f80f854b (127.21.205.193:43669), 07e5e073f6734ed7af8211fdbfb3baac (127.21.205.194:44547)
I20260925 14:46:12.447717 750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.442433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:41391 (local address 127.21.205.194:44547)
0925 14:46:12.442470 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.442472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.444533 (+ 2061us) server_negotiation.cc:407] Connection header received
0925 14:46:12.444647 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.444648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.444660 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.444706 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.444914 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.445589 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.446743 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.447313 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.447363 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.447374 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.447605 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.447652 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:46:12.447896 416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "b7a25f67076246d69f9057fb6e883ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f48c1a4451f4b5a86635009f80f854b"
I20260925 14:46:12.447943 760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.442236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:44547 (local address 127.21.205.195:41391)
0925 14:46:12.444464 (+ 2228us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.444477 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.444545 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.444754 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.444755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.444762 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.444815 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.444816 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.445682 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.445683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.446644 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.446647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.447436 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.447437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.447513 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.447517 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.447835 (+ 318us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.447879 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2206,"thread_start_us":90,"threads_started":1}
I20260925 14:46:12.447976 416 raft_consensus.cc:3037] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:12.448225 552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "b7a25f67076246d69f9057fb6e883ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e5e073f6734ed7af8211fdbfb3baac"
I20260925 14:46:12.448242 553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "b7a25f67076246d69f9057fb6e883ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e5e073f6734ed7af8211fdbfb3baac" is_pre_election: true
I20260925 14:46:12.448371 552 raft_consensus.cc:3037] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:12.448962 416 raft_consensus.cc:2454] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a25f67076246d69f9057fb6e883ce7 in term 1.
I20260925 14:46:12.449132 552 raft_consensus.cc:2454] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7a25f67076246d69f9057fb6e883ce7 in term 1.
I20260925 14:46:12.449169 620 leader_election.cc:302] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f48c1a4451f4b5a86635009f80f854b, b7a25f67076246d69f9057fb6e883ce7; no voters:
I20260925 14:46:12.449287 758 raft_consensus.cc:2781] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:12.449443 758 raft_consensus.cc:686] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 1 LEADER]: Becoming Leader. State: Replica: b7a25f67076246d69f9057fb6e883ce7, State: Running, Role: LEADER
I20260925 14:46:12.449548 758 consensus_queue.cc:237] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } }
I20260925 14:46:12.450454 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.446808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:33407 (local address 127.21.205.194:32879)
0925 14:46:12.446893 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.446903 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.446942 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.447184 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.447185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.447186 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.447251 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.447253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.448163 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.448164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.449376 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.449379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.450190 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.450191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.450291 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.450296 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.450353 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.450394 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 14:46:12.450496 32735 catalog_manager.cc:5495] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7a25f67076246d69f9057fb6e883ce7 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "b7a25f67076246d69f9057fb6e883ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7a25f67076246d69f9057fb6e883ce7" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33407 } health_report { overall_health: HEALTHY } } }
I20260925 14:46:12.450628 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.447033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:32879 (local address 127.21.205.195:33407)
0925 14:46:12.447069 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.447071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.447077 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:46:12.447095 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.447096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.447103 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.447180 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.447311 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.447958 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.449606 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.450102 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.450181 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.450189 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.450539 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.450579 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:46:12.450913 688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "07e5e073f6734ed7af8211fdbfb3baac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7a25f67076246d69f9057fb6e883ce7" is_pre_election: true
I20260925 14:46:12.453023 748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.447006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:55723 (local address 127.21.205.193:43669)
0925 14:46:12.447041 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.447043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.450412 (+ 3369us) server_negotiation.cc:407] Connection header received
0925 14:46:12.450503 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.450504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.450514 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.450560 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:12.450756 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.451366 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.452272 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.452619 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.452715 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.452724 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.452923 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.452961 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:46:12.453038 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.446808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43669 (local address 127.21.205.194:55723)
0925 14:46:12.450348 (+ 3540us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.450359 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:12.450440 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:12.450561 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:12.450561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:12.450563 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:12.450623 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.450625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.451459 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.451460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.452183 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:12.452185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.452717 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.452718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:12.452794 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.452805 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 14:46:12.452914 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 14:46:12.452954 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3521,"mutex_wait_us":10,"thread_start_us":88,"threads_started":1}
I20260925 14:46:12.453318 416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd79aac34f094eb8b1de11280a9fbae0" candidate_uuid: "07e5e073f6734ed7af8211fdbfb3baac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f48c1a4451f4b5a86635009f80f854b" is_pre_election: true
I20260925 14:46:12.453418 416 raft_consensus.cc:2379] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07e5e073f6734ed7af8211fdbfb3baac in current term 1: Already voted for candidate b7a25f67076246d69f9057fb6e883ce7 in this term.
I20260925 14:46:12.453708 484 leader_election.cc:302] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07e5e073f6734ed7af8211fdbfb3baac; no voters: 7f48c1a4451f4b5a86635009f80f854b, b7a25f67076246d69f9057fb6e883ce7
I20260925 14:46:12.453840 761 raft_consensus.cc:2726] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 14:46:12.461583 751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.459403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.21.205.195:33407)
0925 14:46:12.459462 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:12.459464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:12.459472 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:46:12.459520 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:12.459521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:12.459538 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:12.459602 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:46:12.459714 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.460200 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.460934 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:12.461115 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:12.461166 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:12.461294 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:46:12.461395 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:46:12.461458 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:12.461489 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 14:46:12.461505 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:46:12.463552 552 raft_consensus.cc:1261] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 1 FOLLOWER]: Refusing update from remote peer b7a25f67076246d69f9057fb6e883ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:46:12.463600 416 raft_consensus.cc:1261] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 1 FOLLOWER]: Refusing update from remote peer b7a25f67076246d69f9057fb6e883ce7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:46:12.463946 758 consensus_queue.cc:1035] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f48c1a4451f4b5a86635009f80f854b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:46:12.464120 763 consensus_queue.cc:1035] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e5e073f6734ed7af8211fdbfb3baac" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:46:12.469619 32735 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:46:12.469751 32735 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:46:12.470564 32735 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94efae300ee74985acba380f1a3e1199: Sending DeleteTablet for 3 replicas of tablet bd79aac34f094eb8b1de11280a9fbae0
I20260925 14:46:12.470911 385 tablet_service.cc:1514] Processing DeleteTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:12 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:44366
I20260925 14:46:12.470911 668 tablet_service.cc:1514] Processing DeleteTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:12 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:53724
I20260925 14:46:12.471159 777 tablet_replica.cc:330] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b: stopping tablet replica
I20260925 14:46:12.471174 532 tablet_service.cc:1514] Processing DeleteTablet for tablet bd79aac34f094eb8b1de11280a9fbae0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:12 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260925 14:46:12.471305 777 raft_consensus.cc:2229] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:46:12.471433 777 raft_consensus.cc:2258] T bd79aac34f094eb8b1de11280a9fbae0 P 7f48c1a4451f4b5a86635009f80f854b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:46:12.471434 778 tablet_replica.cc:330] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: stopping tablet replica
I20260925 14:46:12.471504 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 329
I20260925 14:46:12.472226 776 tablet_replica.cc:330] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: stopping tablet replica
I20260925 14:46:12.472329 776 raft_consensus.cc:2229] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:46:12.472093 778 raft_consensus.cc:2229] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:46:12.472460 776 raft_consensus.cc:2258] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:46:12.472589 778 raft_consensus.cc:2258] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:46:12.473104 778 ts_tablet_manager.cc:1905] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:46:12.473448 776 ts_tablet_manager.cc:1905] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:46:12.475208 778 ts_tablet_manager.cc:1918] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:46:12.475208 776 ts_tablet_manager.cc:1918] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:46:12.475302 776 log.cc:1196] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/bd79aac34f094eb8b1de11280a9fbae0
I20260925 14:46:12.475302 778 log.cc:1196] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/bd79aac34f094eb8b1de11280a9fbae0
I20260925 14:46:12.475615 776 ts_tablet_manager.cc:1939] T bd79aac34f094eb8b1de11280a9fbae0 P b7a25f67076246d69f9057fb6e883ce7: Deleting consensus metadata
I20260925 14:46:12.475615 778 ts_tablet_manager.cc:1939] T bd79aac34f094eb8b1de11280a9fbae0 P 07e5e073f6734ed7af8211fdbfb3baac: Deleting consensus metadata
I20260925 14:46:12.476249 32718 catalog_manager.cc:4841] TS 07e5e073f6734ed7af8211fdbfb3baac (127.21.205.194:44547): tablet bd79aac34f094eb8b1de11280a9fbae0 (table test-table [id=8b1ca89dc65647f2b404c615bb7f84ea]) successfully deleted
I20260925 14:46:12.476280 32720 catalog_manager.cc:4841] TS b7a25f67076246d69f9057fb6e883ce7 (127.21.205.195:33407): tablet bd79aac34f094eb8b1de11280a9fbae0 (table test-table [id=8b1ca89dc65647f2b404c615bb7f84ea]) successfully deleted
W20260925 14:46:12.477779 32718 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:43669 (error 108)
I20260925 14:46:12.478386 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:12.478238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43669 (local address 127.0.0.1:44378)
0925 14:46:12.478305 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:12.478354 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 14:46:12.478420 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 468
W20260925 14:46:12.478526 32718 catalog_manager.cc:4584] TS 7f48c1a4451f4b5a86635009f80f854b (127.21.205.193:43669): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd79aac34f094eb8b1de11280a9fbae0: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43669: connect: Connection refused (error 111)
I20260925 14:46:12.485885 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 604
I20260925 14:46:12.491309 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 32702
2026-09-25T14:46:12Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1899 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[782](info): set up 2 sockets
Sep 25 14:46:12 dist-test-slave-d7lw krb5kdc[782](info): commencing operation
krb5kdc: starting...
W20260925 14:46:13.551393 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:46:13 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:46:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:46:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangeradmin_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangeradmin_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangerlookup_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangerlookup_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
I20260925 14:46:13.714471 22327 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY8Adyw/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/postgres -l logfile start
2026-09-25 14:46:16.165 UTC [813] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 14:46:16.165 UTC [813] LOG: listening on IPv4 address "127.21.205.212", port 56371
2026-09-25 14:46:16.168 UTC [813] LOG: listening on Unix socket "/tmp/.s.PGSQL.56371"
2026-09-25 14:46:16.189 UTC [816] LOG: database system was shut down at 2026-09-25 14:46:14 UTC
2026-09-25 14:46:16.192 UTC [813] LOG: database system is ready to accept connections
I20260925 14:46:17.155057 22327 mini_postgres.cc:96] Postgres bound to 56371
2026-09-25 14:46:17.161 UTC [824] FATAL: database "slave" does not exist
127.21.205.212:56371 - accepting connections
I20260925 14:46:17.161549 22327 mini_ranger.cc:161] Starting Ranger...
I20260925 14:46:17.176403 22327 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 14:46:17.628381 22327 mini_ranger.cc:87] Created ranger Postgres database
I20260925 14:46:17.628464 22327 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin
2026-09-25 14:46:17,966 [I] DB FLAVOR :POSTGRES
2026-09-25 14:46:17,966 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 14:46:17,966 [I] Checking connection..
2026-09-25 14:46:17,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 14:46:18,234 [I] Checking connection passed.
2026-09-25 14:46:18,235 [I] --------- Verifying version history table ---------
2026-09-25 14:46:18,235 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:46:18,467 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 14:46:18,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:46:18,689 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 14:46:18,690 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 14:46:18,690 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 14:46:18.911 UTC [889] WARNING: there is no transaction in progress
2026-09-25 14:46:18,917 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 14:46:18,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:46:19,169 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 14:46:19,170 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 14:46:19,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 14:46:19,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 14:46:19,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-d7lw.c.gcp-upstream.internal','N') ;"
2026-09-25 14:46:19,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 14:46:20,113 [I] Table x_portal_user does not exist in database ranger
2026-09-25 14:46:20,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 14:46:20,356 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 14:46:20,357 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 14:46:20,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 14:46:20,827 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 14:46:20,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 14:46:21.733 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.748 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.765 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.779 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.794 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.810 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.862 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.868 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.882 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.892 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.901 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.916 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.924 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.930 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.936 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:21.942 UTC [1000] WARNING: there is no transaction in progress
2026-09-25 14:46:22,531 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 14:46:22,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 14:46:22,764 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 14:46:22,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 14:46:23,006 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 14:46:23,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 14:46:23,264 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 14:46:23,265 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 14:46:23,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 14:46:23,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-d7lw.c.gcp-upstream.internal';"
2026-09-25 14:46:24,047 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 14:46:24,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 14:46:24,316 [I] DB_PATCHES have already been applied
I20260925 14:46:24.322681 22327 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY8Adyw/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 14:46:24.322755 22327 mini_ranger.cc:193] Using host: 127.21.205.212
I20260925 14:46:24.326023 22327 mini_ranger.cc:227] Ranger admin URL: http://127.21.205.212:48919
Sep 25, 2026 2:46:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:46:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:46:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:46:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:46:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:46:25 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 14:46:25.451843 22327 mini_ranger.cc:160] Time spent starting Ranger: real 8.290s user 0.005s sys 0.005s
Sep 25, 2026 2:46:46 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:46:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 2:46:46 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:46:46 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:46:47 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 14:46:48.259184 22327 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangerkms_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/rangerkms_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.205.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.205.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 14:46:48 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347608, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 14:46:48.351920 22327 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 14:46:48.361469 22327 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 14:46:49.130131 22327 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 14:46:49.130218 22327 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms
I20260925 14:46:49.130236 22327 mini_ranger_kms.cc:227] Using postgres at 127.21.205.212:56371
2026-09-25 14:46:49,314 [I] DB FLAVOR :POSTGRES
2026-09-25 14:46:49,314 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 14:46:49,314 [I] Checking connection
2026-09-25 14:46:49,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 14:46:49,543 [I] connection success
2026-09-25 14:46:49,544 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 14:46:49,544 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 14:46:49,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 14:46:49,799 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 14:46:49,799 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 14:46:49,799 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 14:46:49,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:56371/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 14:46:50,069 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 14:46:50.516355 22327 mini_ranger_kms.cc:310] Created kms service
I20260925 14:46:50.676578 22327 mini_ranger_kms.cc:326] Created kudu user
I20260925 14:46:50.718596 22327 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 14:46:50 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347608, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:51.090452 22327 mini_ranger_kms.cc:384] Added ranger policy
I20260925 14:46:51.090571 22327 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 14:46:51.090608 22327 mini_ranger_kms.cc:240] Using host: 127.21.205.212
I20260925 14:46:51.093777 22327 mini_ranger_kms.cc:276] Ranger KMS PID: 1182
I20260925 14:46:51.093896 22327 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.205.212:38623
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:46:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 14:46:53 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790347613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:53.402464 22327 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.051s user 0.002s sys 0.165s
I20260925 14:46:53.402582 22327 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 14:46:53.402665 22327 mini_ranger_kms.cc:401] 127.21.205.212:38623/kms/v1/keys
Sep 25 14:46:53 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
Sep 25 14:46:54 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347613, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
Sep 25 14:46:56 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:46:56.510267 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43443
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:43443
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:38623/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:56.594517 1234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:56.594758 1234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:56.594781 1234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:56.596096 1234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:46:56.596148 1234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:56.596163 1234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:56.596175 1234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:46:56.596187 1234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:46:56.597913 1234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.205.212:38623/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:43443
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:56.598192 1234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:56.598457 1234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:56.603821 1239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:56.603824 1234 server_base.cc:1034] running on GCE node
W20260925 14:46:56.603852 1242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:56.604221 1240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:56.604534 1234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:56.604859 1234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:56.606030 1234 hybrid_clock.cc:648] HybridClock initialized: now 1790347616606017 us; error 53 us; skew 500 ppm
Sep 25 14:46:56 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:56.608873 1234 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:46:56.609345 1234 webserver.cc:460] Webserver started at http://127.21.205.254:42849/ using document root <none> and password file <none>
I20260925 14:46:56.609599 1234 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:56.609666 1234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:56.609825 1234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:46:56 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347616, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:56.663206 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "76560c8d94994337a1130a6c13828594"
format_stamp: "Formatted at 2026-09-25 14:46:56 on dist-test-slave-d7lw"
server_key: "0be3556745e98992a65d27f10318f176"
server_key_iv: "5275e6ca05272f9799b05a151117605a"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:56.663733 1234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "76560c8d94994337a1130a6c13828594"
format_stamp: "Formatted at 2026-09-25 14:46:56 on dist-test-slave-d7lw"
server_key: "0be3556745e98992a65d27f10318f176"
server_key_iv: "5275e6ca05272f9799b05a151117605a"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:56.665692 1234 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 14:46:56.681231 1249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:56.681696 1234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:46:56.681820 1234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "76560c8d94994337a1130a6c13828594"
format_stamp: "Formatted at 2026-09-25 14:46:56 on dist-test-slave-d7lw"
server_key: "0be3556745e98992a65d27f10318f176"
server_key_iv: "5275e6ca05272f9799b05a151117605a"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:56.681941 1234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:56.700973 1234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:56.703255 1234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:56.703420 1234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:56.708727 1234 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:43443
I20260925 14:46:56.708747 1303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:43443 every 8 connection(s)
I20260925 14:46:56.709184 1234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:46:56.709929 1304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:56.712487 1304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: Bootstrap starting.
I20260925 14:46:56.713204 1304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:56.713548 1304 log.cc:824] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:56.714249 1304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: No bootstrap required, opened a new log
I20260925 14:46:56.716415 1304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } }
I20260925 14:46:56.716570 1304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:56.716593 1304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76560c8d94994337a1130a6c13828594, State: Initialized, Role: FOLLOWER
I20260925 14:46:56.716789 1304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } }
I20260925 14:46:56.716888 1304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:46:56.716895 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 1234
I20260925 14:46:56.716933 1304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:46:56.716996 1304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:56.717023 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:46:56.717216 22327 external_mini_cluster.cc:1428] Setting key 21c97f4d6fc3a3b88c770ddb2932db5c
I20260925 14:46:56.717773 1304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } }
I20260925 14:46:56.717911 1304 leader_election.cc:302] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76560c8d94994337a1130a6c13828594; no voters:
I20260925 14:46:56.718132 1304 leader_election.cc:288] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:46:56.718209 1309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:56.718410 1309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [term 1 LEADER]: Becoming Leader. State: Replica: 76560c8d94994337a1130a6c13828594, State: Running, Role: LEADER
I20260925 14:46:56.718484 1304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:46:56.718542 1309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } }
I20260925 14:46:56.718947 1309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76560c8d94994337a1130a6c13828594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } } }
I20260925 14:46:56.719090 1309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:56.719264 1309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76560c8d94994337a1130a6c13828594. Latest consensus state: current_term: 1 leader_uuid: "76560c8d94994337a1130a6c13828594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76560c8d94994337a1130a6c13828594" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43443 } } }
I20260925 14:46:56.719364 1309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594 [sys.catalog]: This master's current role is: LEADER
I20260925 14:46:56.719514 1317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:46:56.720109 1317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:46:56.720359 1304 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 25 14:46:56 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:56.729235 1317 catalog_manager.cc:1347] Generated new cluster ID: 481609614c054b158719d4f3a037d16b
I20260925 14:46:56.729354 1317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:46:56.732358 1308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:56.717746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.21.205.254:43443)
0925 14:46:56.718002 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:56.718014 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:56.718054 (+ 40us) server_negotiation.cc:407] Connection header received
0925 14:46:56.718188 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:56.718196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:56.718270 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:56.719226 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:56.719339 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:56.720931 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:56.722527 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:56.724505 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:56.724699 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:56.727638 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:56.727643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:56.727643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:56.727656 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:56.730253 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:56.730662 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:56.730665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:56.730665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:56.730770 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:56.730965 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:56.730965 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:56.730966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:56.731274 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:56.731852 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:56.731919 (+ 67us) server_negotiation.cc:299] Negotiation successful
0925 14:46:56.732098 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":85,"threads_started":1}
I20260925 14:46:56.750737 1317 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:46:56.751662 1317 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:46:56.759593 1317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: Generated new TSK 0
I20260925 14:46:56.759898 1317 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:46:57.494524 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:38623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:57.620637 1350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:57.620941 1350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:57.620983 1350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:57.622884 1350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:57.622970 1350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:57.623014 1350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:46:57.625451 1350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.205.212:38623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:57.625764 1350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:57.626057 1350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:57.631861 1355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:57.631834 1356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:57.632090 1350 server_base.cc:1034] running on GCE node
W20260925 14:46:57.631857 1358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:57.632412 1350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:57.632705 1350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:57.633877 1350 hybrid_clock.cc:648] HybridClock initialized: now 1790347617633864 us; error 32 us; skew 500 ppm
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:57.636701 1350 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:46:57.637228 1350 webserver.cc:460] Webserver started at http://127.21.205.193:37531/ using document root <none> and password file <none>
I20260925 14:46:57.637471 1350 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:57.637548 1350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:57.637714 1350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:57.653653 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "c6483f35748f4edbb211926f5dc3f4f5"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "1f036bdb3a994c2bcdb80869bf5e2576"
server_key_iv: "46d49152ef9b9f7612eb290a9b679534"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.654222 1350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "c6483f35748f4edbb211926f5dc3f4f5"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "1f036bdb3a994c2bcdb80869bf5e2576"
server_key_iv: "46d49152ef9b9f7612eb290a9b679534"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.656605 1350 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:57.670998 1365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.671271 1350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:57.671437 1350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "c6483f35748f4edbb211926f5dc3f4f5"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "1f036bdb3a994c2bcdb80869bf5e2576"
server_key_iv: "46d49152ef9b9f7612eb290a9b679534"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.671571 1350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:57.682746 1350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:57.685385 1350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:57.685601 1350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:57.685899 1350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:57.686305 1350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:57.686378 1350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.686439 1350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:57.686475 1350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.694680 1350 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:41323
I20260925 14:46:57.694948 1482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:41323 every 8 connection(s)
I20260925 14:46:57.695254 1350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:46:57.701659 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 1350
I20260925 14:46:57.701773 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:46:57.701988 22327 external_mini_cluster.cc:1428] Setting key 352941f110b36601e792224395740f5c
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:57.708443 1485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:57.696084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:43443 (local address 127.21.205.193:52635)
0925 14:46:57.696512 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:57.696560 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:57.697108 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:57.697515 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:57.697520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:57.697782 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:57.697992 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:57.697996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.698758 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.698759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:57.700088 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:57.700091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.700666 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.700667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:57.700814 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:57.701458 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:57.701468 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:57.704122 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:57.706473 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:57.706484 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:57.706487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:57.706858 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:57.707106 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:57.707107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:57.707107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:57.707267 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:57.707618 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:57.707625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:57.707857 (+ 232us) client_negotiation.cc:769] Sending connection context
0925 14:46:57.707981 (+ 124us) client_negotiation.cc:240] Negotiation successful
0925 14:46:57.708174 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":92,"threads_started":1}
I20260925 14:46:57.709175 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:57.696401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:52635 (local address 127.21.205.254:43443)
0925 14:46:57.696532 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:57.696534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:57.696653 (+ 119us) server_negotiation.cc:407] Connection header received
0925 14:46:57.697227 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:57.697229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:57.697244 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:57.697359 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:57.698107 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.698638 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.700196 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.700509 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.700592 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:57.704246 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:57.704248 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:57.704248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:57.704257 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:57.706367 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:57.706956 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:57.706957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:57.706958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:57.707026 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:57.707348 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:57.707349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:57.707349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:57.707505 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:57.707587 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:57.708946 (+ 1359us) server_negotiation.cc:299] Negotiation successful
0925 14:46:57.709050 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260925 14:46:57.710021 1483 heartbeater.cc:347] Connected to a master server at 127.21.205.254:43443
I20260925 14:46:57.710126 1483 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:57.710330 1483 heartbeater.cc:511] Master 127.21.205.254:43443 requested a full tablet report, sending...
I20260925 14:46:57.710953 1267 ts_manager.cc:194] Registered new tserver with Master: c6483f35748f4edbb211926f5dc3f4f5 (127.21.205.193:41323)
I20260925 14:46:57.711925 1267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:52635
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:46:57.743026 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:38623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:57.853597 1489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:57.853861 1489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:57.853904 1489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:57.855182 1489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:57.855258 1489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:57.855304 1489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:46:57.857008 1489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.205.212:38623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:57.857287 1489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:57.857555 1489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:57.862708 1499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:57.862751 1496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:57.862984 1497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:57.863775 1489 server_base.cc:1034] running on GCE node
I20260925 14:46:57.864075 1489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:57.864364 1489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:57.865525 1489 hybrid_clock.cc:648] HybridClock initialized: now 1790347617865500 us; error 42 us; skew 500 ppm
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:57.868479 1489 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:46:57.869055 1489 webserver.cc:460] Webserver started at http://127.21.205.194:42569/ using document root <none> and password file <none>
I20260925 14:46:57.869308 1489 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:57.869385 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:57.869534 1489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:57.882165 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "d3fef039f5a64e2db962f806d604d3db"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "7e78631fb6f74c5f2eebdb9136e35a1d"
server_key_iv: "c0de08f424a3b19968867cafb2fab9f3"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.882791 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "d3fef039f5a64e2db962f806d604d3db"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "7e78631fb6f74c5f2eebdb9136e35a1d"
server_key_iv: "c0de08f424a3b19968867cafb2fab9f3"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.890522 1489 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.002s
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:57.918649 1506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.918970 1489 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.000s sys 0.001s
I20260925 14:46:57.919100 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "d3fef039f5a64e2db962f806d604d3db"
format_stamp: "Formatted at 2026-09-25 14:46:57 on dist-test-slave-d7lw"
server_key: "7e78631fb6f74c5f2eebdb9136e35a1d"
server_key_iv: "c0de08f424a3b19968867cafb2fab9f3"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:57.919214 1489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:57.939402 1489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:57.942333 1489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:57.942576 1489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:57.942906 1489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:57.943320 1489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:57.943405 1489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.943472 1489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:57.943522 1489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:57.950819 1489 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:39371
I20260925 14:46:57.950976 1621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:39371 every 8 connection(s)
I20260925 14:46:57.951427 1489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:46:57 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:57.960583 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 1489
I20260925 14:46:57.960721 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:46:57.960983 22327 external_mini_cluster.cc:1428] Setting key 545249359cdd667504c1f1bb1cc97037
I20260925 14:46:57.962813 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:57.952601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:44677 (local address 127.21.205.254:43443)
0925 14:46:57.952667 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:57.952669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:57.952701 (+ 32us) server_negotiation.cc:407] Connection header received
0925 14:46:57.953429 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:57.953430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:57.953444 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:57.953505 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:57.954364 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.955178 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.956594 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.956944 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.957010 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:57.959226 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:57.959227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:57.959228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:57.959236 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:57.960825 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:57.961587 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:57.961589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:57.961589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:57.961656 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:57.961975 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:57.961977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:57.961977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:57.962142 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:57.962216 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:57.962556 (+ 340us) server_negotiation.cc:299] Negotiation successful
0925 14:46:57.962667 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 14:46:57.962936 1624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:57.952322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:43443 (local address 127.21.205.194:44677)
0925 14:46:57.952535 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:57.952591 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:57.953295 (+ 704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:57.953606 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:57.953611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:57.953886 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:57.954255 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:57.954264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.955298 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.955299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:57.956502 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:57.956505 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:57.957035 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:57.957036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:57.957153 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:57.957803 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:57.957814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:57.959125 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:57.960916 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:57.960924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:57.960928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:57.961255 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:57.961736 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:57.961737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:57.961738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:57.961901 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:57.962216 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:57.962221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:57.962394 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 14:46:57.962511 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 14:46:57.962677 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":84,"threads_started":1}
I20260925 14:46:57.963565 1622 heartbeater.cc:347] Connected to a master server at 127.21.205.254:43443
I20260925 14:46:57.963697 1622 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:57.963938 1622 heartbeater.cc:511] Master 127.21.205.254:43443 requested a full tablet report, sending...
I20260925 14:46:57.964406 1267 ts_manager.cc:194] Registered new tserver with Master: d3fef039f5a64e2db962f806d604d3db (127.21.205.194:39371)
I20260925 14:46:57.964929 1267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:44677
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:46:57.987109 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:38623/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40847
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:46:58.085492 1628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:46:58.085803 1628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:46:58.085942 1628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:46:58.087262 1628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:46:58.087337 1628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:46:58.087386 1628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:46:58.089061 1628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40847
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.205.212:38623/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:43443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:46:58.089350 1628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:46:58.089622 1628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:46:58.095427 1634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:46:58.095508 1636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:58.096030 1628 server_base.cc:1034] running on GCE node
W20260925 14:46:58.096271 1633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:46:58.096517 1628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:46:58.096787 1628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:46:58.097952 1628 hybrid_clock.cc:648] HybridClock initialized: now 1790347618097924 us; error 40 us; skew 500 ppm
Sep 25 14:46:58 dist-test-slave-d7lw krb5kdc[782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:46:58.100656 1628 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:46:58.101181 1628 webserver.cc:460] Webserver started at http://127.21.205.195:41449/ using document root <none> and password file <none>
I20260925 14:46:58.101436 1628 fs_manager.cc:359] Metadata directory not provided
I20260925 14:46:58.101506 1628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:46:58.101663 1628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:46:58 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:58.113870 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "39b25252b7ab46e997a853ebec14254e"
format_stamp: "Formatted at 2026-09-25 14:46:58 on dist-test-slave-d7lw"
server_key: "1b3b010e916bc367c813bfdba80c412a"
server_key_iv: "5e44807c10679c76d950b6c75fa1bd9c"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:58.114354 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "39b25252b7ab46e997a853ebec14254e"
format_stamp: "Formatted at 2026-09-25 14:46:58 on dist-test-slave-d7lw"
server_key: "1b3b010e916bc367c813bfdba80c412a"
server_key_iv: "5e44807c10679c76d950b6c75fa1bd9c"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:58.120658 1628 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.003s
I20260925 14:46:58.133780 1643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:58.134014 1628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:46:58.134130 1628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "39b25252b7ab46e997a853ebec14254e"
format_stamp: "Formatted at 2026-09-25 14:46:58 on dist-test-slave-d7lw"
server_key: "1b3b010e916bc367c813bfdba80c412a"
server_key_iv: "5e44807c10679c76d950b6c75fa1bd9c"
server_key_version: "kuduclusterkey@0"
I20260925 14:46:58.134239 1628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:46:58.158387 1628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:46:58.161098 1628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:46:58.161255 1628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:46:58.161500 1628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:46:58.161880 1628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:46:58.161931 1628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:58.161967 1628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:46:58.161988 1628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:46:58.168871 1628 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:41335
I20260925 14:46:58.168939 1757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:41335 every 8 connection(s)
I20260925 14:46:58.169315 1628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:46:58.173899 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 1628
I20260925 14:46:58.174021 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:46:58.174196 22327 external_mini_cluster.cc:1428] Setting key 31112b24bb41e94de23995f182266b00
Sep 25 14:46:58 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347618, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:46:58.180305 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.170106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:45927 (local address 127.21.205.254:43443)
0925 14:46:58.170155 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.170157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.170436 (+ 279us) server_negotiation.cc:407] Connection header received
0925 14:46:58.171235 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.171236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.171249 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.171318 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:58.172127 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.172924 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.174524 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.174823 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.174874 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.177395 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:58.177397 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:58.177397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:58.177404 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:58.178743 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:58.179243 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:58.179244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:58.179244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:58.179299 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:58.179565 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:58.179566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:58.179567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:58.179694 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:58.179734 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.180113 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.180206 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:46:58.180943 1760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.170087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:43443 (local address 127.21.205.195:45927)
0925 14:46:58.170264 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.170313 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.171125 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.171404 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.171409 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.171655 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:46:58.171993 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.172000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.173052 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.173053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.174414 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.174417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.174969 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.174970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.175116 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.175733 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:46:58.175744 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:46:58.177281 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:46:58.178875 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:58.178882 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:58.178884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:58.179157 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:58.179365 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:46:58.179366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:46:58.179366 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:46:58.179494 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:46:58.179798 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:46:58.179803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:46:58.179959 (+ 156us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.180545 (+ 586us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.180705 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260925 14:46:58.181515 1758 heartbeater.cc:347] Connected to a master server at 127.21.205.254:43443
I20260925 14:46:58.181612 1758 heartbeater.cc:464] Registering TS with master...
I20260925 14:46:58.181794 1758 heartbeater.cc:511] Master 127.21.205.254:43443 requested a full tablet report, sending...
I20260925 14:46:58.182209 1267 ts_manager.cc:194] Registered new tserver with Master: 39b25252b7ab46e997a853ebec14254e (127.21.205.195:41335)
I20260925 14:46:58.182695 1267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:45927
I20260925 14:46:58.185773 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:46:58.194525 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.187587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.21.205.254:43443)
0925 14:46:58.187630 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.187632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.187819 (+ 187us) server_negotiation.cc:407] Connection header received
0925 14:46:58.187889 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.187890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.187899 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.187946 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:46:58.188661 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.189136 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.190192 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.190383 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.190446 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.191952 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:46:58.191953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:46:58.191953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:46:58.191960 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:46:58.193438 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:58.193834 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:58.193835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:58.193836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:58.193885 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:46:58.194091 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:46:58.194091 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:46:58.194091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:46:58.194188 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:46:58.194263 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.194402 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.194458 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:46:58.196444 1267 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:46:58.198205 1267 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:46:58.206429 1776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.201457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35964 (local address 127.21.205.193:41323)
0925 14:46:58.201584 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.201590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.202403 (+ 813us) server_negotiation.cc:407] Connection header received
0925 14:46:58.202452 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.202455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.202470 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.202540 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:58.202745 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203814 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.205124 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.205898 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.205991 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.206062 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.206312 (+ 250us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.206352 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260925 14:46:58.206414 1772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.200962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:41335 (local address 127.0.0.1:35398)
0925 14:46:58.201891 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.201900 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.201937 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.202195 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.202196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.202198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:58.202247 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.202249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203527 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.203528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.204996 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.204997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.206123 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.206124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.206181 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.206197 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.206266 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.206329 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":6,"thread_start_us":121,"threads_started":1}
I20260925 14:46:58.206661 1775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.201299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.21.205.195:41335)
0925 14:46:58.202024 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.202029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.202036 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:46:58.202092 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.202094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.202111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.202180 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:58.202327 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203436 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.205128 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.205961 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.206034 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.206095 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.206541 (+ 446us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.206586 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":77,"threads_started":1}
I20260925 14:46:58.207046 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.201239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:41323 (local address 127.0.0.1:35964)
0925 14:46:58.202281 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.202291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.202338 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.202636 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.202637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.202638 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:58.202672 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.202673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203901 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.203902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.204994 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.204997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.206099 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.206101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.206181 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.206197 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.206266 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.206978 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":666,"thread_start_us":69,"threads_started":1}
I20260925 14:46:58.207522 1774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.200955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39371 (local address 127.0.0.1:47716)
0925 14:46:58.201369 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.201414 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.201487 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.201788 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.201792 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.201795 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:58.202177 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.202180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203576 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.203577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.205379 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.205382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.207318 (+ 1936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.207319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.207376 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.207379 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.207421 (+ 42us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.207457 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":43,"thread_start_us":124,"threads_started":1}
I20260925 14:46:58.207415 1692 tablet_service.cc:1467] Processing CreateTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 (DEFAULT_TABLE table=test-table [id=8642f8e338da41df85696ba3926b9dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:58.207760 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.201053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47716 (local address 127.21.205.194:39371)
0925 14:46:58.201598 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.201603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.201614 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:46:58.201663 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.201665 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.201685 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.201780 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:58.202281 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.203454 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.205523 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.207138 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.207205 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.207258 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.207628 (+ 370us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.207685 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
I20260925 14:46:58.207855 1692 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363db06ff8f34cc18c0a0193bae01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:58.207916 1416 tablet_service.cc:1467] Processing CreateTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 (DEFAULT_TABLE table=test-table [id=8642f8e338da41df85696ba3926b9dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:58.208215 1416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363db06ff8f34cc18c0a0193bae01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:58.215619 1547 tablet_service.cc:1467] Processing CreateTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 (DEFAULT_TABLE table=test-table [id=8642f8e338da41df85696ba3926b9dbc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:46:58.216002 1547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 363db06ff8f34cc18c0a0193bae01a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:46:58.216596 1778 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: Bootstrap starting.
I20260925 14:46:58.217633 1778 tablet_bootstrap.cc:654] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:58.218040 1778 log.cc:824] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:58.218458 1777 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Bootstrap starting.
I20260925 14:46:58.219520 1777 tablet_bootstrap.cc:654] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:58.219930 1777 log.cc:824] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:58.220777 1777 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: No bootstrap required, opened a new log
I20260925 14:46:58.220837 1777 ts_tablet_manager.cc:1397] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:58.222352 1780 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Bootstrap starting.
I20260925 14:46:58.222829 1777 raft_consensus.cc:348] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.223012 1777 raft_consensus.cc:374] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:58.223050 1777 raft_consensus.cc:729] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b25252b7ab46e997a853ebec14254e, State: Initialized, Role: FOLLOWER
I20260925 14:46:58.223165 1777 consensus_queue.cc:260] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.223299 1780 tablet_bootstrap.cc:654] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Neither blocks nor log segments found. Creating new log.
I20260925 14:46:58.223445 1777 ts_tablet_manager.cc:1428] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:46:58.223623 1780 log.cc:824] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Log is configured to *not* fsync() on all Append() calls
I20260925 14:46:58.223865 1758 heartbeater.cc:503] Master 127.21.205.254:43443 was elected leader, sending a full tablet report...
I20260925 14:46:58.224370 1778 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: No bootstrap required, opened a new log
I20260925 14:46:58.224435 1778 ts_tablet_manager.cc:1397] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260925 14:46:58.226039 1778 raft_consensus.cc:348] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.226207 1778 raft_consensus.cc:374] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:58.226249 1778 raft_consensus.cc:729] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6483f35748f4edbb211926f5dc3f4f5, State: Initialized, Role: FOLLOWER
I20260925 14:46:58.226377 1778 consensus_queue.cc:260] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.226624 1778 ts_tablet_manager.cc:1428] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:46:58.226635 1483 heartbeater.cc:503] Master 127.21.205.254:43443 was elected leader, sending a full tablet report...
I20260925 14:46:58.227532 1780 tablet_bootstrap.cc:492] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: No bootstrap required, opened a new log
I20260925 14:46:58.227600 1780 ts_tablet_manager.cc:1397] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260925 14:46:58.229094 1780 raft_consensus.cc:348] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.229238 1780 raft_consensus.cc:374] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:46:58.229279 1780 raft_consensus.cc:729] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3fef039f5a64e2db962f806d604d3db, State: Initialized, Role: FOLLOWER
I20260925 14:46:58.229395 1780 consensus_queue.cc:260] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.229650 1780 ts_tablet_manager.cc:1428] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260925 14:46:58.229689 1622 heartbeater.cc:503] Master 127.21.205.254:43443 was elected leader, sending a full tablet report...
I20260925 14:46:58.312803 1784 raft_consensus.cc:482] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:46:58.312952 1784 raft_consensus.cc:504] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.313371 1784 leader_election.cc:288] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3fef039f5a64e2db962f806d604d3db (127.21.205.194:39371), 39b25252b7ab46e997a853ebec14254e (127.21.205.195:41335)
I20260925 14:46:58.317788 1775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.313730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:51983 (local address 127.21.205.195:41335)
0925 14:46:58.313776 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.313778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.313956 (+ 178us) server_negotiation.cc:407] Connection header received
0925 14:46:58.314051 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.314052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.314062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.314096 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:58.314360 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.315141 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.316806 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.317370 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.317415 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.317424 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.317652 (+ 228us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.317696 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:46:58.317889 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.313743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:59211 (local address 127.21.205.194:39371)
0925 14:46:58.313784 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.313786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.313935 (+ 149us) server_negotiation.cc:407] Connection header received
0925 14:46:58.314050 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.314051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.314072 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.314103 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:46:58.314360 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.315127 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.316990 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.317526 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.317565 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.317573 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.317778 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.317821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:46:58.318138 1787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.313508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39371 (local address 127.21.205.193:59211)
0925 14:46:58.313865 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.313877 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.313965 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.314172 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.314172 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.314175 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:58.314279 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.314281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.315257 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.315259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.316900 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.316903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.317628 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.317629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.317702 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.317705 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.317998 (+ 293us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.318047 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":122,"thread_start_us":119,"threads_started":1}
I20260925 14:46:58.318420 1786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.313542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:41335 (local address 127.21.205.193:51983)
0925 14:46:58.313904 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.313912 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:46:58.313979 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:46:58.314171 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:46:58.314172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:46:58.314174 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:46:58.314280 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.314281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.315266 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.315266 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.316675 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:46:58.316679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.317484 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.317485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:46:58.317554 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.317558 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:46:58.318169 (+ 611us) client_negotiation.cc:240] Negotiation successful
0925 14:46:58.318344 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":63,"thread_start_us":91,"threads_started":1}
I20260925 14:46:58.318511 1576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363db06ff8f34cc18c0a0193bae01a96" candidate_uuid: "c6483f35748f4edbb211926f5dc3f4f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3fef039f5a64e2db962f806d604d3db" is_pre_election: true
I20260925 14:46:58.318693 1576 raft_consensus.cc:2454] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6483f35748f4edbb211926f5dc3f4f5 in term 0.
I20260925 14:46:58.318801 1712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363db06ff8f34cc18c0a0193bae01a96" candidate_uuid: "c6483f35748f4edbb211926f5dc3f4f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b25252b7ab46e997a853ebec14254e" is_pre_election: true
I20260925 14:46:58.318953 1712 raft_consensus.cc:2454] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6483f35748f4edbb211926f5dc3f4f5 in term 0.
I20260925 14:46:58.318945 1369 leader_election.cc:302] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c6483f35748f4edbb211926f5dc3f4f5, d3fef039f5a64e2db962f806d604d3db; no voters:
I20260925 14:46:58.319073 1784 raft_consensus.cc:2781] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:46:58.319123 1784 raft_consensus.cc:482] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:46:58.319145 1784 raft_consensus.cc:3037] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:58.319839 1784 raft_consensus.cc:504] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.319965 1784 leader_election.cc:288] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [CANDIDATE]: Term 1 election: Requested vote from peers d3fef039f5a64e2db962f806d604d3db (127.21.205.194:39371), 39b25252b7ab46e997a853ebec14254e (127.21.205.195:41335)
I20260925 14:46:58.320122 1712 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363db06ff8f34cc18c0a0193bae01a96" candidate_uuid: "c6483f35748f4edbb211926f5dc3f4f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b25252b7ab46e997a853ebec14254e"
I20260925 14:46:58.320151 1576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "363db06ff8f34cc18c0a0193bae01a96" candidate_uuid: "c6483f35748f4edbb211926f5dc3f4f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3fef039f5a64e2db962f806d604d3db"
I20260925 14:46:58.320191 1712 raft_consensus.cc:3037] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:58.320221 1576 raft_consensus.cc:3037] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:46:58.321012 1576 raft_consensus.cc:2454] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6483f35748f4edbb211926f5dc3f4f5 in term 1.
I20260925 14:46:58.321012 1712 raft_consensus.cc:2454] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6483f35748f4edbb211926f5dc3f4f5 in term 1.
I20260925 14:46:58.321192 1368 leader_election.cc:302] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39b25252b7ab46e997a853ebec14254e, c6483f35748f4edbb211926f5dc3f4f5; no voters:
I20260925 14:46:58.321350 1784 raft_consensus.cc:2781] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:46:58.321516 1784 raft_consensus.cc:686] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [term 1 LEADER]: Becoming Leader. State: Replica: c6483f35748f4edbb211926f5dc3f4f5, State: Running, Role: LEADER
I20260925 14:46:58.321611 1784 consensus_queue.cc:237] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } }
I20260925 14:46:58.323653 1267 catalog_manager.cc:5495] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6483f35748f4edbb211926f5dc3f4f5 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "c6483f35748f4edbb211926f5dc3f4f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6483f35748f4edbb211926f5dc3f4f5" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 41323 } health_report { overall_health: HEALTHY } } }
I20260925 14:46:58.329977 1776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.327593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.21.205.193:41323)
0925 14:46:58.327725 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 14:46:58.327727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:46:58.327732 (+ 5us) server_negotiation.cc:407] Connection header received
0925 14:46:58.327764 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:46:58.327765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:46:58.327814 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:46:58.327877 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:46:58.327998 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.328441 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.329199 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:46:58.329417 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:46:58.329491 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:46:58.329662 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:46:58.329763 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:46:58.329853 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 14:46:58.329873 (+ 20us) server_negotiation.cc:299] Negotiation successful
0925 14:46:58.329892 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260925 14:46:58.331547 1712 raft_consensus.cc:1261] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 1 FOLLOWER]: Refusing update from remote peer c6483f35748f4edbb211926f5dc3f4f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:46:58.331856 1576 raft_consensus.cc:1261] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 1 FOLLOWER]: Refusing update from remote peer c6483f35748f4edbb211926f5dc3f4f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:46:58.331879 1784 consensus_queue.cc:1035] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b25252b7ab46e997a853ebec14254e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 41335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:46:58.332096 1784 consensus_queue.cc:1035] T 363db06ff8f34cc18c0a0193bae01a96 P c6483f35748f4edbb211926f5dc3f4f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3fef039f5a64e2db962f806d604d3db" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:46:58.334017 1792 mvcc.cc:205] Tried to move back new op lower bound from 7333263844683972608 to 7333263844645912576. Current Snapshot: MvccSnapshot[applied={T|T < 7333263844683972608}]
I20260925 14:46:58.334882 1793 mvcc.cc:205] Tried to move back new op lower bound from 7333263844683972608 to 7333263844645912576. Current Snapshot: MvccSnapshot[applied={T|T < 7333263844683972608}]
I20260925 14:46:58.338039 1267 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:46:58.338131 1267 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:46:58.338902 1267 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 76560c8d94994337a1130a6c13828594: Sending DeleteTablet for 3 replicas of tablet 363db06ff8f34cc18c0a0193bae01a96
I20260925 14:46:58.339298 1547 tablet_service.cc:1514] Processing DeleteTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:47716
I20260925 14:46:58.339572 1416 tablet_service.cc:1514] Processing DeleteTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:35964
I20260925 14:46:58.339986 1692 tablet_service.cc:1514] Processing DeleteTablet for tablet 363db06ff8f34cc18c0a0193bae01a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:46:58 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:35398
I20260925 14:46:58.340440 1794 mvcc.cc:205] Tried to move back new op lower bound from 7333263844683972608 to 7333263844645912576. Current Snapshot: MvccSnapshot[applied={T|T < 7333263844683972608}]
I20260925 14:46:58.340602 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 1350
I20260925 14:46:58.340616 1802 tablet_replica.cc:330] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: stopping tablet replica
I20260925 14:46:58.341403 1804 tablet_replica.cc:330] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: stopping tablet replica
I20260925 14:46:58.341578 1802 raft_consensus.cc:2229] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:46:58.341722 1804 raft_consensus.cc:2229] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:46:58.341832 1804 raft_consensus.cc:2258] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:46:58.341974 1802 raft_consensus.cc:2258] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:46:58.342871 1802 ts_tablet_manager.cc:1905] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:46:58.343493 1804 ts_tablet_manager.cc:1905] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:46:58.345475 1802 ts_tablet_manager.cc:1918] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:46:58.347327 1804 ts_tablet_manager.cc:1918] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:46:58.347445 1804 log.cc:1196] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/363db06ff8f34cc18c0a0193bae01a96
I20260925 14:46:58.347796 1804 ts_tablet_manager.cc:1939] T 363db06ff8f34cc18c0a0193bae01a96 P 39b25252b7ab46e997a853ebec14254e: Deleting consensus metadata
I20260925 14:46:58.347916 1802 log.cc:1196] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/363db06ff8f34cc18c0a0193bae01a96
W20260925 14:46:58.348397 1252 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:41323 (error 108)
I20260925 14:46:58.348579 1802 ts_tablet_manager.cc:1939] T 363db06ff8f34cc18c0a0193bae01a96 P d3fef039f5a64e2db962f806d604d3db: Deleting consensus metadata
I20260925 14:46:58.348946 1252 catalog_manager.cc:4841] TS d3fef039f5a64e2db962f806d604d3db (127.21.205.194:39371): tablet 363db06ff8f34cc18c0a0193bae01a96 (table test-table [id=8642f8e338da41df85696ba3926b9dbc]) successfully deleted
I20260925 14:46:58.349272 1771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:46:58.349163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:41323 (local address 127.0.0.1:35972)
0925 14:46:58.349203 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 14:46:58.349239 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:41323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260925 14:46:58.349277 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 1489
W20260925 14:46:58.350456 1252 catalog_manager.cc:4584] TS c6483f35748f4edbb211926f5dc3f4f5 (127.21.205.193:41323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 363db06ff8f34cc18c0a0193bae01a96: Network error: Client connection negotiation failed: client connection to 127.21.205.193:41323: connect: Connection refused (error 111)
I20260925 14:46:58.350852 1251 catalog_manager.cc:4841] TS 39b25252b7ab46e997a853ebec14254e (127.21.205.195:41335): tablet 363db06ff8f34cc18c0a0193bae01a96 (table test-table [id=8642f8e338da41df85696ba3926b9dbc]) successfully deleted
I20260925 14:46:58.356227 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 1628
Sep 25 14:46:58 dist-test-slave-d7lw krb5kdc[782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347617, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:46:58.383445 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 1234
I20260925 14:46:58.389047 22327 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 14:46:58.733754 22327 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 14:46:58.733882 22327 mini_ranger.cc:66] Stopping Ranger...
I20260925 14:46:59.079149 22327 mini_ranger.cc:68] Stopped Ranger
2026-09-25 14:46:59.079 UTC [813] LOG: received smart shutdown request
2026-09-25 14:46:59.082 UTC [813] LOG: background worker "logical replication launcher" (PID 822) exited with exit code 1
2026-09-25 14:46:59.083 UTC [817] LOG: shutting down
2026-09-25 14:46:59.258 UTC [813] LOG: database system is shut down
2026-09-25T14:46:59Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46871 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:46:59 dist-test-slave-d7lw krb5kdc[1814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:46:59 dist-test-slave-d7lw krb5kdc[1814](info): set up 2 sockets
Sep 25 14:46:59 dist-test-slave-d7lw krb5kdc[1814](info): commencing operation
krb5kdc: starting...
W20260925 14:47:00.402629 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:47:00 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:47:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangeradmin_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangeradmin_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangerlookup_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangerlookup_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
I20260925 14:47:00.553615 22327 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskY8Adyw/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/postgres -l logfile start
2026-09-25 14:47:02.878 UTC [1845] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 14:47:02.878 UTC [1845] LOG: listening on IPv4 address "127.21.205.212", port 39981
2026-09-25 14:47:02.881 UTC [1845] LOG: listening on Unix socket "/tmp/.s.PGSQL.39981"
2026-09-25 14:47:02.901 UTC [1848] LOG: database system was shut down at 2026-09-25 14:47:01 UTC
2026-09-25 14:47:02.904 UTC [1845] LOG: database system is ready to accept connections
I20260925 14:47:03.866806 22327 mini_postgres.cc:96] Postgres bound to 39981
2026-09-25 14:47:03.872 UTC [1856] FATAL: database "slave" does not exist
127.21.205.212:39981 - accepting connections
I20260925 14:47:03.873485 22327 mini_ranger.cc:161] Starting Ranger...
I20260925 14:47:03.887378 22327 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 14:47:04.326284 22327 mini_ranger.cc:87] Created ranger Postgres database
I20260925 14:47:04.326349 22327 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin
2026-09-25 14:47:04,578 [I] DB FLAVOR :POSTGRES
2026-09-25 14:47:04,579 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 14:47:04,579 [I] Checking connection..
2026-09-25 14:47:04,579 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 14:47:04,801 [I] Checking connection passed.
2026-09-25 14:47:04,801 [I] --------- Verifying version history table ---------
2026-09-25 14:47:04,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:47:05,023 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 14:47:05,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:47:05,260 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 14:47:05,261 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 14:47:05,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 14:47:05.512 UTC [1921] WARNING: there is no transaction in progress
2026-09-25 14:47:05,518 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 14:47:05,518 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 14:47:05,747 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 14:47:05,747 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 14:47:05,748 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 14:47:05,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 14:47:06,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-d7lw.c.gcp-upstream.internal','N') ;"
2026-09-25 14:47:06,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 14:47:06,692 [I] Table x_portal_user does not exist in database ranger
2026-09-25 14:47:06,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 14:47:06,936 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 14:47:06,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 14:47:07,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 14:47:07,405 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 14:47:07,405 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 14:47:08.307 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.322 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.336 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.350 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.364 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.381 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.432 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.439 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.452 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.461 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.470 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.482 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.492 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.498 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.504 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:08.510 UTC [2030] WARNING: there is no transaction in progress
2026-09-25 14:47:09,083 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 14:47:09,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 14:47:09,322 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 14:47:09,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 14:47:09,574 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 14:47:09,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 14:47:09,828 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 14:47:09,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 14:47:10,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 14:47:10,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-d7lw.c.gcp-upstream.internal';"
2026-09-25 14:47:10,555 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 14:47:10,555 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 14:47:10,803 [I] DB_PATCHES have already been applied
I20260925 14:47:10.809742 22327 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskY8Adyw/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 14:47:10.809826 22327 mini_ranger.cc:193] Using host: 127.21.205.212
I20260925 14:47:10.813145 22327 mini_ranger.cc:227] Ranger admin URL: http://127.21.205.212:56227
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 2:47:11 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 14:47:11.904911 22327 mini_ranger.cc:160] Time spent starting Ranger: real 8.031s user 0.000s sys 0.008s
Sep 25, 2026 2:47:30 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 2:47:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 25, 2026 2:47:30 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 2:47:30 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 2:47:31 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 14:47:32.249528 22327 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.205.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.205.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangerkms_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.205.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/rangerkms_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.205.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.205.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.205.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/HTTP_127.21.205.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 14:47:32 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 14:47:32.326292 22327 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 14:47:32.334905 22327 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 14:47:33.127180 22327 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 14:47:33.127261 22327 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms
I20260925 14:47:33.127280 22327 mini_ranger_kms.cc:227] Using postgres at 127.21.205.212:39981
2026-09-25 14:47:33,290 [I] DB FLAVOR :POSTGRES
2026-09-25 14:47:33,290 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 14:47:33,290 [I] Checking connection
2026-09-25 14:47:33,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 14:47:33,527 [I] connection success
2026-09-25 14:47:33,527 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 14:47:33,527 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 14:47:33,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 14:47:33,795 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 14:47:33,796 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 14:47:33,796 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 14:47:33,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.205.212:39981/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 14:47:34,049 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 14:47:34.499485 22327 mini_ranger_kms.cc:310] Created kms service
I20260925 14:47:34.680665 22327 mini_ranger_kms.cc:326] Created kudu user
I20260925 14:47:34.721099 22327 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 14:47:34 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347652, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:35.096181 22327 mini_ranger_kms.cc:384] Added ranger policy
I20260925 14:47:35.096295 22327 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms:/tmp/dist-test-taskY8Adyw/build/release/bin/postgresql.jar:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskY8Adyw/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 14:47:35.096315 22327 mini_ranger_kms.cc:240] Using host: 127.21.205.212
I20260925 14:47:35.100922 22327 mini_ranger_kms.cc:276] Ranger KMS PID: 2214
I20260925 14:47:35.100999 22327 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.205.212:49225
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 2:47:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 14:47:37 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790347657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:37.337522 22327 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.011s user 0.003s sys 0.144s
I20260925 14:47:37.337625 22327 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 14:47:37.337677 22327 mini_ranger_kms.cc:401] 127.21.205.212:49225/kms/v1/keys
Sep 25 14:47:37 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
Sep 25 14:47:37 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347657, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.205.212@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
Sep 25 14:47:40 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:47:40.142707 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:32805
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:40.235110 2267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:40.235399 2267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:40.235435 2267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:40.236773 2267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:40.236824 2267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:40.236927 2267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:40.236961 2267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:40.236989 2267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:40.238484 2267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.205.212:49225/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:32805
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:40.238752 2267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:40.238986 2267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:40.244874 2273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:40.244947 2275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:40.245041 2267 server_base.cc:1034] running on GCE node
W20260925 14:47:40.245138 2272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:40.245424 2267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:40.245760 2267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:40.246924 2267 hybrid_clock.cc:648] HybridClock initialized: now 1790347660246905 us; error 38 us; skew 500 ppm
Sep 25 14:47:40 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:40.252990 2267 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:40.253391 2267 webserver.cc:460] Webserver started at http://127.21.205.254:45291/ using document root <none> and password file <none>
I20260925 14:47:40.253602 2267 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:40.253659 2267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:40.253808 2267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:47:40 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347660, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:40.303843 2267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "125dbb071a664166af56580eecbe16f8"
format_stamp: "Formatted at 2026-09-25 14:47:40 on dist-test-slave-d7lw"
server_key: "d4291fafa21bfe31ea26d9ff8f7a545d"
server_key_iv: "130884a5831b97d526c706d3f9c0d92b"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:40.304297 2267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "125dbb071a664166af56580eecbe16f8"
format_stamp: "Formatted at 2026-09-25 14:47:40 on dist-test-slave-d7lw"
server_key: "d4291fafa21bfe31ea26d9ff8f7a545d"
server_key_iv: "130884a5831b97d526c706d3f9c0d92b"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:40.306126 2267 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:40.320415 2282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:40.320644 2267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:40.320751 2267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "125dbb071a664166af56580eecbe16f8"
format_stamp: "Formatted at 2026-09-25 14:47:40 on dist-test-slave-d7lw"
server_key: "d4291fafa21bfe31ea26d9ff8f7a545d"
server_key_iv: "130884a5831b97d526c706d3f9c0d92b"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:40.320842 2267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:40.336899 2267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:40.338930 2267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:40.339089 2267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:40.344102 2267 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:32805
I20260925 14:47:40.344177 2334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:32805 every 8 connection(s)
I20260925 14:47:40.344568 2267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:40.345265 2335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:40.347460 2335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Bootstrap starting.
I20260925 14:47:40.348119 2335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:40.348444 2335 log.cc:824] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:40.348481 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2267
I20260925 14:47:40.348563 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:47:40.348750 22327 external_mini_cluster.cc:1428] Setting key fe0335858831d41bc00cf3d5a5507e77
I20260925 14:47:40.349247 2335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: No bootstrap required, opened a new log
I20260925 14:47:40.351755 2335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:40.351945 2335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:40.351984 2335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125dbb071a664166af56580eecbe16f8, State: Initialized, Role: FOLLOWER
I20260925 14:47:40.352128 2335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:40.352224 2335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:40.352265 2335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:40.352317 2335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:40.352941 2335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:40.353049 2335 leader_election.cc:302] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 125dbb071a664166af56580eecbe16f8; no voters:
I20260925 14:47:40.353243 2335 leader_election.cc:288] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:47:40.353269 2340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:40.353467 2340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 LEADER]: Becoming Leader. State: Replica: 125dbb071a664166af56580eecbe16f8, State: Running, Role: LEADER
I20260925 14:47:40.353610 2335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:40.353590 2340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:40.355093 2340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "125dbb071a664166af56580eecbe16f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:47:40.355230 2340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:40.355417 2340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 125dbb071a664166af56580eecbe16f8. Latest consensus state: current_term: 1 leader_uuid: "125dbb071a664166af56580eecbe16f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:47:40.355520 2340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:40.356254 2335 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 14:47:40.356639 2354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 25 14:47:40 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:40.360091 2354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:40.364193 2339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:40.349262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58416 (local address 127.21.205.254:32805)
0925 14:47:40.349447 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:40.349455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:40.349501 (+ 46us) server_negotiation.cc:407] Connection header received
0925 14:47:40.350444 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:40.350454 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:40.350512 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:40.350655 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:40.351464 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:40.352868 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:40.353971 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:40.354629 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:40.354740 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:40.358521 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:40.358574 (+ 53us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:40.358574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:40.358708 (+ 134us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:40.361810 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:40.362217 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:40.362222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:40.362223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:40.362328 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:40.362567 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:40.362567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:40.362568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:40.363017 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:40.363542 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:40.363608 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 14:47:40.363915 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":62,"threads_started":1}
I20260925 14:47:40.369913 2354 catalog_manager.cc:1347] Generated new cluster ID: 8d0601677f5b4f1da5395c609d49fc67
I20260925 14:47:40.369973 2354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:47:40.388828 2354 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:47:40.389660 2354 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:40.402845 2354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Generated new TSK 0
I20260925 14:47:40.403134 2354 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:47:41.097075 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:41.181432 2382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:41.181644 2382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:41.181682 2382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:41.182790 2382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:41.182855 2382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:41.182897 2382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:47:41.184649 2382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:41.184907 2382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:41.185137 2382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:41.190910 2388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.191092 2382 server_base.cc:1034] running on GCE node
W20260925 14:47:41.191200 2387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.191305 2390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.191532 2382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:41.191817 2382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:41.192966 2382 hybrid_clock.cc:648] HybridClock initialized: now 1790347661192955 us; error 36 us; skew 500 ppm
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:41.195451 2382 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:47:41.196028 2382 webserver.cc:460] Webserver started at http://127.21.205.193:45045/ using document root <none> and password file <none>
I20260925 14:47:41.196313 2382 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:41.196399 2382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:41.196534 2382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:41.208484 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "f346cbc4045b4227a67871a920c7bf86"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "2b3d152f0fcb648f4e46b5509788ed38"
server_key_iv: "d629a9a690f32c9834cf2aece244eda3"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.208932 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "f346cbc4045b4227a67871a920c7bf86"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "2b3d152f0fcb648f4e46b5509788ed38"
server_key_iv: "d629a9a690f32c9834cf2aece244eda3"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.211417 2382 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:41.226094 2397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.226315 2382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:41.226432 2382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "f346cbc4045b4227a67871a920c7bf86"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "2b3d152f0fcb648f4e46b5509788ed38"
server_key_iv: "d629a9a690f32c9834cf2aece244eda3"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.226521 2382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:41.239208 2382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:41.241492 2382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:41.241652 2382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:41.241885 2382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:41.242251 2382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:41.242302 2382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.242332 2382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:41.242358 2382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.249519 2382 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:43249
I20260925 14:47:41.249596 2512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:43249 every 8 connection(s)
I20260925 14:47:41.249982 2382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:47:41.252099 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2382
I20260925 14:47:41.252254 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:47:41.252499 22327 external_mini_cluster.cc:1428] Setting key 01173f0525e14ea5646c9f7abda2c712
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:41.261067 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.250720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:35005 (local address 127.21.205.254:32805)
0925 14:47:41.251929 (+ 1209us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:41.251931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:41.251940 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:47:41.251968 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:41.251968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:41.251980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:41.252806 (+ 826us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:41.252849 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.253627 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.255108 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.255431 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.255945 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.257922 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:41.257924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:41.257924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:41.257931 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:41.259652 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.260115 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.260116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.260116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.260178 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.260436 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.260437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.260438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.260553 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:41.260628 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:41.260891 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 14:47:41.260972 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1171,"thread_start_us":100,"threads_started":1}
I20260925 14:47:41.261256 2515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.250685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.193:35005)
0925 14:47:41.250957 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:41.250998 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:41.251613 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:41.252158 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:41.252163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:41.252389 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:41.252693 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.252710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.253733 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.253734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.255008 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.255011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.255532 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.255533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.255663 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.256284 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:41.256294 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:41.257811 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:41.259739 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.259746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.259748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.260047 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.260256 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.260256 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.260257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.260362 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.260620 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:41.260626 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:41.260758 (+ 132us) client_negotiation.cc:769] Sending connection context
0925 14:47:41.260859 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 14:47:41.261004 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":87,"threads_started":1}
I20260925 14:47:41.261736 2513 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:41.261833 2513 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:41.262013 2513 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:41.262645 2299 ts_manager.cc:194] Registered new tserver with Master: f346cbc4045b4227a67871a920c7bf86 (127.21.205.193:43249)
I20260925 14:47:41.263449 2299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:35005
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:47:41.286295 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:41.399722 2520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:41.400102 2520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:41.400156 2520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:41.401877 2520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:41.401960 2520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 14:47:41.402002 2520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:47:41.404826 2520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:41.405165 2520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:41.405468 2520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:47:41.412284 2520 server_base.cc:1034] running on GCE node
W20260925 14:47:41.412423 2527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.415474 2530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.415691 2528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.415980 2520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:41.416266 2520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:41.417429 2520 hybrid_clock.cc:648] HybridClock initialized: now 1790347661417407 us; error 35 us; skew 500 ppm
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:41.421584 2520 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:47:41.422129 2520 webserver.cc:460] Webserver started at http://127.21.205.194:41937/ using document root <none> and password file <none>
I20260925 14:47:41.422402 2520 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:41.422462 2520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:41.422588 2520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:41.438395 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "5a079e5149c044f5838fab7d78510173"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "dec6c411119afda0c581a46cbb8dcce9"
server_key_iv: "e16277002944e521c8ad51e839e8fe7f"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.438832 2520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "5a079e5149c044f5838fab7d78510173"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "dec6c411119afda0c581a46cbb8dcce9"
server_key_iv: "e16277002944e521c8ad51e839e8fe7f"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.447969 2520 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260925 14:47:41.460572 2537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.463930 2520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 14:47:41.464044 2520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "5a079e5149c044f5838fab7d78510173"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "dec6c411119afda0c581a46cbb8dcce9"
server_key_iv: "e16277002944e521c8ad51e839e8fe7f"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.464150 2520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:41.493894 2520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:41.496438 2520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:41.496639 2520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:41.496969 2520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:41.497372 2520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:41.497454 2520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.497532 2520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:41.497586 2520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.506235 2520 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:42319
I20260925 14:47:41.506718 2520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:47:41.508160 2654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:42319 every 8 connection(s)
I20260925 14:47:41.516592 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2520
I20260925 14:47:41.516697 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:47:41.516896 22327 external_mini_cluster.cc:1428] Setting key f4ecee3b3bb0d78aefab8e4691a7e6c3
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:41.524268 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.507882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:37841 (local address 127.21.205.254:32805)
0925 14:47:41.507944 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:41.507947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:41.514068 (+ 6121us) server_negotiation.cc:407] Connection header received
0925 14:47:41.514649 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:41.514650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:41.514668 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:41.514716 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:41.515568 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.516379 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.517892 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.518191 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.518238 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.520873 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:41.520874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:41.520875 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:41.520882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:41.522585 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.523069 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.523070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.523071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.523129 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.523437 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.523438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.523438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.523592 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:41.523644 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:41.524056 (+ 412us) server_negotiation.cc:299] Negotiation successful
0925 14:47:41.524149 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:47:41.524804 2657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.507730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.194:37841)
0925 14:47:41.513919 (+ 6189us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:41.513968 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:41.514509 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:41.514852 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:41.514856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:41.515120 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:41.515446 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.515453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.516507 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.516507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.517790 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.517793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.518362 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.518363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.518506 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.519138 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:41.519149 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:41.520761 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:41.522671 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.522677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.522679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.522972 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.523204 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.523204 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.523205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.523353 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.523695 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:41.523701 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:41.523897 (+ 196us) client_negotiation.cc:769] Sending connection context
0925 14:47:41.524405 (+ 508us) client_negotiation.cc:240] Negotiation successful
0925 14:47:41.524572 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5955,"thread_start_us":82,"threads_started":1}
I20260925 14:47:41.525543 2655 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:41.525640 2655 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:41.525832 2655 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:41.526274 2299 ts_manager.cc:194] Registered new tserver with Master: 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319)
I20260925 14:47:41.526746 2299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:37841
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:47:41.556420 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:41.679946 2660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:41.680326 2660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:41.680418 2660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:41.682139 2660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:41.682246 2660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:41.682332 2660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:47:41.684571 2660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:41.685087 2660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:41.685412 2660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:41.692562 2669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.693321 2666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.693636 2667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.693959 2660 server_base.cc:1034] running on GCE node
I20260925 14:47:41.694240 2660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:41.694517 2660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:41.697813 2660 hybrid_clock.cc:648] HybridClock initialized: now 1790347661697792 us; error 28 us; skew 500 ppm
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:41.701943 2660 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:47:41.702518 2660 webserver.cc:460] Webserver started at http://127.21.205.195:36461/ using document root <none> and password file <none>
I20260925 14:47:41.702826 2660 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:41.702939 2660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:41.703110 2660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:41.723192 2660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "a5c317052f1240b197cde33f3ce50747"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "07647779f95702588097084c0a651790"
server_key_iv: "3e2c9c576b012725a52ef6c1feca70c4"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.723819 2660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "a5c317052f1240b197cde33f3ce50747"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "07647779f95702588097084c0a651790"
server_key_iv: "3e2c9c576b012725a52ef6c1feca70c4"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.725713 2660 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:41.741420 2676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.741626 2660 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260925 14:47:41.741724 2660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "a5c317052f1240b197cde33f3ce50747"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "07647779f95702588097084c0a651790"
server_key_iv: "3e2c9c576b012725a52ef6c1feca70c4"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.741838 2660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:41.761518 2660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:41.763950 2660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:41.764150 2660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:41.764422 2660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:41.764806 2660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:41.764885 2660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.764972 2660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:41.765030 2660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.772504 2660 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:45697
I20260925 14:47:41.772873 2660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:47:41.773495 2790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:45697 every 8 connection(s)
I20260925 14:47:41.775326 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2660
I20260925 14:47:41.775449 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:47:41.775636 22327 external_mini_cluster.cc:1428] Setting key 2d4e5d53d37d2872aabd2266204f3dba
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:41.782923 2516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.773647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:51267 (local address 127.21.205.254:32805)
0925 14:47:41.773691 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:41.773693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:41.773903 (+ 210us) server_negotiation.cc:407] Connection header received
0925 14:47:41.774532 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:41.774533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:41.774545 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:41.774612 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:41.775851 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.776591 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.777658 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.777872 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.777949 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.780036 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:41.780037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:41.780038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:41.780045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:41.781409 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.781908 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.781909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.781910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.781979 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:41.782257 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:41.782258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:41.782258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:41.782382 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:41.782468 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:41.782750 (+ 282us) server_negotiation.cc:299] Negotiation successful
0925 14:47:41.782836 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:47:41.783123 2793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:41.773646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.195:51267)
0925 14:47:41.773783 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:41.773823 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:41.774429 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:41.774688 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:41.774693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:41.774886 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:41.775092 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.775097 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.776686 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.776688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.777555 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:41.777558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:41.777981 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:41.777982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:41.778099 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:41.778618 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:41.778626 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:41.779947 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:41.781509 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.781516 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.781518 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.781743 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.782051 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:41.782052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:41.782053 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:41.782185 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:41.782458 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:41.782465 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:41.782608 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 14:47:41.782714 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 14:47:41.782872 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
I20260925 14:47:41.783685 2791 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:41.783841 2791 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:41.784029 2791 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:41.784435 2299 ts_manager.cc:194] Registered new tserver with Master: a5c317052f1240b197cde33f3ce50747 (127.21.205.195:45697)
I20260925 14:47:41.784802 2299 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:51267
I20260925 14:47:41.787163 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:47:41.787959 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2382
I20260925 14:47:41.795753 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2520
I20260925 14:47:41.802050 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2660
I20260925 14:47:41.811057 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2267
I20260925 14:47:41.820668 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--webserver_interface=127.21.205.254
--webserver_port=45291
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:32805
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:41.900354 2794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:41.900583 2794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:41.900619 2794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:41.901770 2794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:41.901830 2794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:41.901854 2794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:41.901881 2794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:41.901911 2794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:41.903420 2794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:32805
--ranger_config_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=45291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:41.903679 2794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:41.903940 2794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:41.909739 2800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.909833 2794 server_base.cc:1034] running on GCE node
W20260925 14:47:41.909765 2802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:41.909973 2799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:41.910214 2794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:41.910573 2794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:41.911742 2794 hybrid_clock.cc:648] HybridClock initialized: now 1790347661911720 us; error 36 us; skew 500 ppm
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:41.914397 2794 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:41.914796 2794 webserver.cc:460] Webserver started at http://127.21.205.254:45291/ using document root <none> and password file <none>
I20260925 14:47:41.915022 2794 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:41.915097 2794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 14:47:41 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347661, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:41.959111 2794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:41.959908 2810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:41.960124 2794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:41.960242 2794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "125dbb071a664166af56580eecbe16f8"
format_stamp: "Formatted at 2026-09-25 14:47:40 on dist-test-slave-d7lw"
server_key: "d4291fafa21bfe31ea26d9ff8f7a545d"
server_key_iv: "130884a5831b97d526c706d3f9c0d92b"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:41.960623 2794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:41.968863 2794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:41.970459 2794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:41.970623 2794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:41.975013 2794 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:32805
I20260925 14:47:41.975044 2863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:32805 every 8 connection(s)
I20260925 14:47:41.975440 2794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:41.975870 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2794
I20260925 14:47:41.976174 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:43249
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:32805
--webserver_port=45045
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:41.976886 2864 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:47:41.977531 2864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Bootstrap starting.
I20260925 14:47:41.985675 2864 log.cc:824] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:41.987378 2864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:47:41.987658 2864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Bootstrap complete.
I20260925 14:47:41.990048 2864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:41.990299 2864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125dbb071a664166af56580eecbe16f8, State: Initialized, Role: FOLLOWER
I20260925 14:47:41.990492 2864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:41.990572 2864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:41.990624 2864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:41.990676 2864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 1 FOLLOWER]: Advancing to term 2
I20260925 14:47:41.991655 2864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:41.991773 2864 leader_election.cc:302] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 125dbb071a664166af56580eecbe16f8; no voters:
I20260925 14:47:41.991971 2864 leader_election.cc:288] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 14:47:41.992003 2869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 14:47:41.992108 2869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [term 2 LEADER]: Becoming Leader. State: Replica: 125dbb071a664166af56580eecbe16f8, State: Running, Role: LEADER
I20260925 14:47:41.992249 2864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:41.992285 2869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:47:41.992566 2869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 125dbb071a664166af56580eecbe16f8. Latest consensus state: current_term: 2 leader_uuid: "125dbb071a664166af56580eecbe16f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:47:41.992628 2869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:41.993016 2874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:41.993088 2871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "125dbb071a664166af56580eecbe16f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "125dbb071a664166af56580eecbe16f8" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:47:41.993294 2871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:41.993709 2874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:41.993889 2874 catalog_manager.cc:1259] Loaded cluster ID: 8d0601677f5b4f1da5395c609d49fc67
I20260925 14:47:41.993932 2874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:47:41.994087 2864 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 14:47:41.997624 2874 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:41.998198 2874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Loaded TSK: 0
I20260925 14:47:41.998503 2874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:47:42.084852 2867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:42.085105 2867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:42.085139 2867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:42.086735 2867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:42.086795 2867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:42.086838 2867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:47:42.088835 2867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:43249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=45045
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:42.089067 2867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:42.089318 2867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:42.095619 2904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:42.095935 2906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:42.096171 2867 server_base.cc:1034] running on GCE node
W20260925 14:47:42.096371 2903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:42.096607 2867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:42.096886 2867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:42.098058 2867 hybrid_clock.cc:648] HybridClock initialized: now 1790347662098039 us; error 34 us; skew 500 ppm
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:42.101989 2867 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:47:42.102491 2867 webserver.cc:460] Webserver started at http://127.21.205.193:45045/ using document root <none> and password file <none>
I20260925 14:47:42.102887 2867 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:42.103016 2867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:42.130632 2867 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:42.131752 2914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.131927 2867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:42.132010 2867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "f346cbc4045b4227a67871a920c7bf86"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "2b3d152f0fcb648f4e46b5509788ed38"
server_key_iv: "d629a9a690f32c9834cf2aece244eda3"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:42.132539 2867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:42.155114 2867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:42.157483 2867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:42.157652 2867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:42.157894 2867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:42.158255 2867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:42.158304 2867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.158344 2867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:42.158378 2867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.165753 2867 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:43249
I20260925 14:47:42.165930 3030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:43249 every 8 connection(s)
I20260925 14:47:42.166208 2867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:47:42.174603 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 2867
I20260925 14:47:42.174916 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:42319
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:32805
--webserver_port=41937
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:42.192051 3033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.166987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.193:47935)
0925 14:47:42.167629 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:42.167674 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:42.168585 (+ 911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:42.169287 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:42.169291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:42.169598 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:42.169907 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.169913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.171677 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.171678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.173115 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.173118 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.174157 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.174158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.174298 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.177294 (+ 2996us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:42.177304 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:42.179020 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:42.187847 (+ 8827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.187856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.187859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.188222 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.188581 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.188583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.188583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.188753 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.191315 (+ 2562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:42.191322 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:42.191491 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 14:47:42.191600 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 14:47:42.191767 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":93,"threads_started":1}
I20260925 14:47:42.192847 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.167393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:47935 (local address 127.21.205.254:32805)
0925 14:47:42.168222 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:42.168232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:42.168294 (+ 62us) server_negotiation.cc:407] Connection header received
0925 14:47:42.168929 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:42.168935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:42.168990 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:42.169121 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:42.170051 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.171543 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.173237 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.173912 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.174023 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.179159 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:42.179164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:42.179165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:42.179177 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:42.183675 (+ 4498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.188356 (+ 4681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.188361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.188362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.188491 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.188848 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.188849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.188849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.189248 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:42.189342 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:42.192519 (+ 3177us) server_negotiation.cc:299] Negotiation successful
0925 14:47:42.192648 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":76,"threads_started":1}
I20260925 14:47:42.193889 3031 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:42.193982 3031 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.194175 3031 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.194993 3031 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:32805 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 14:47:42.195080 3031 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.195142 3031 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:42.195523 3031 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.195593 3031 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.195955 3031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:47:42.299274 3035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:42.299494 3035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:42.299535 3035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:42.300881 3035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:42.300944 3035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:42.300992 3035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:47:42.302632 3035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:42319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=41937
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:42.302906 3035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:42.303229 3035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:42.309072 3043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:42.309105 3041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:42.309193 3040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:42.309511 3035 server_base.cc:1034] running on GCE node
I20260925 14:47:42.309729 3035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:42.310034 3035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:42.311190 3035 hybrid_clock.cc:648] HybridClock initialized: now 1790347662311165 us; error 43 us; skew 500 ppm
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:42.314246 3035 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:47:42.314765 3035 webserver.cc:460] Webserver started at http://127.21.205.194:41937/ using document root <none> and password file <none>
I20260925 14:47:42.315062 3035 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:42.315120 3035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:42.331904 3035 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:42.336263 3051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.336489 3035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260925 14:47:42.336573 3035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "5a079e5149c044f5838fab7d78510173"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "dec6c411119afda0c581a46cbb8dcce9"
server_key_iv: "e16277002944e521c8ad51e839e8fe7f"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:42.337041 3035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:42.358047 3035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:42.360360 3035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:42.360550 3035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:42.360864 3035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:42.361207 3035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:42.361296 3035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.361342 3035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:42.361372 3035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.369300 3035 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:42319
I20260925 14:47:42.369804 3035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:47:42.371966 3166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:42319 every 8 connection(s)
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:42.379740 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3035
I20260925 14:47:42.380054 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.205.212:49225/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:45697
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:32805
--webserver_port=36461
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:37167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:42.385869 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.372734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:42389 (local address 127.21.205.254:32805)
0925 14:47:42.372790 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:42.372793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:42.373048 (+ 255us) server_negotiation.cc:407] Connection header received
0925 14:47:42.373792 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:42.373794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:42.373808 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:42.373844 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:42.374737 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.375559 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.377139 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.377480 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.377527 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.381829 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:42.381831 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:42.381831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:42.381840 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:42.383878 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.384543 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.384544 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.384544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.384610 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.384934 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.384936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.384936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.385116 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:42.385167 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:42.385608 (+ 441us) server_negotiation.cc:299] Negotiation successful
0925 14:47:42.385714 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:47:42.386446 3169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.372712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.194:42389)
0925 14:47:42.372898 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:42.372950 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:42.373644 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:42.374035 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:42.374039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:42.374301 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:42.374610 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.374616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.375690 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.375691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.377023 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.377027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.377647 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.377649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.377797 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.378414 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:42.378424 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:42.381698 (+ 3274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:42.384004 (+ 2306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.384012 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.384015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.384420 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.384695 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.384696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.384697 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.384841 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.385238 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:42.385244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:42.385425 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 14:47:42.386029 (+ 604us) client_negotiation.cc:240] Negotiation successful
0925 14:47:42.386199 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260925 14:47:42.386951 3167 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:42.387075 3167 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.387324 3167 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.388053 3167 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:32805 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 14:47:42.388136 3167 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.388227 3167 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:42.388602 3167 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.388695 3167 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.389025 3167 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:47:42.489637 3170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:42.490008 3170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:42.490115 3170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:42.491837 3170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:42.491935 3170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:42.492018 3170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:47:42.494524 3170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:37167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.205.212:49225/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:45697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=36461
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:42.494786 3170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:42.495069 3170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:42.501155 3177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:42.501220 3179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:42.501163 3176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:42.501840 3170 server_base.cc:1034] running on GCE node
I20260925 14:47:42.502095 3170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:42.502365 3170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:42.503547 3170 hybrid_clock.cc:648] HybridClock initialized: now 1790347662503532 us; error 32 us; skew 500 ppm
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:42.506518 3170 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:47:42.506945 3170 webserver.cc:460] Webserver started at http://127.21.205.195:36461/ using document root <none> and password file <none>
I20260925 14:47:42.507244 3170 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:42.507324 3170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:42.525818 3170 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260925 14:47:42.526947 3187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.527140 3170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:42.527223 3170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "a5c317052f1240b197cde33f3ce50747"
format_stamp: "Formatted at 2026-09-25 14:47:41 on dist-test-slave-d7lw"
server_key: "07647779f95702588097084c0a651790"
server_key_iv: "3e2c9c576b012725a52ef6c1feca70c4"
server_key_version: "kuduclusterkey@0"
I20260925 14:47:42.527676 3170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:42.540731 3170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:42.543074 3170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:42.543215 3170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:42.543428 3170 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:42.543766 3170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:42.543852 3170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.543895 3170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:42.543926 3170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:42.551553 3170 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:45697
I20260925 14:47:42.551620 3301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:45697 every 8 connection(s)
I20260925 14:47:42.552063 3170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:47:42 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347662, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:42.558106 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3170
I20260925 14:47:42.561386 3034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.552847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:46629 (local address 127.21.205.254:32805)
0925 14:47:42.552901 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:42.552903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:42.553145 (+ 242us) server_negotiation.cc:407] Connection header received
0925 14:47:42.553704 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:42.553706 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:42.553718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:42.553765 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:42.554307 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.554812 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.555830 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.556042 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.556119 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.558111 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:42.558112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:42.558113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:42.558120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:42.559981 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.560469 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.560470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.560470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.560528 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.560768 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.560769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.560770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.560884 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:42.560951 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:42.561215 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 14:47:42.561292 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 14:47:42.561537 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.552853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.195:46629)
0925 14:47:42.553022 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:42.553062 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:42.553590 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:42.553830 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:42.553834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:42.554008 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:42.554210 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.554214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.554911 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.554912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.555701 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:42.555702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.556145 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.556147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:42.556267 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.556775 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:42.556783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:42.558000 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:42.560077 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.560083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.560085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.560369 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.560592 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:42.560593 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:42.560594 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:42.560695 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:42.560953 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:42.560960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:42.561085 (+ 125us) client_negotiation.cc:769] Sending connection context
0925 14:47:42.561183 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 14:47:42.561316 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":104,"threads_started":1}
I20260925 14:47:42.562093 3302 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:47:42.562242 3302 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.562419 3302 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.563038 3302 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:32805 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 14:47:42.563117 3302 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.563184 3302 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:42.563472 3302 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:42.563544 3302 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
W20260925 14:47:42.563954 3302 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 14:47:42.567045 3306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:42.558669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.21.205.254:32805)
0925 14:47:42.558790 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:42.558792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:42.558798 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:47:42.558823 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:42.558823 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:42.558833 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:42.558921 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:42.559602 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.560219 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.561670 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:42.561976 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:42.562034 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:42.563657 (+ 1623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:42.563658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:42.563659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:42.563665 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:42.565791 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.566216 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.566218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.566218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.566278 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:42.566515 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:42.566515 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:42.566516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:42.566614 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:42.566761 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:42.566858 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 14:47:42.566950 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
Sep 25 14:47:43 dist-test-slave-d7lw krb5kdc[1814](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790347663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:43.196128 3031 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:43.196280 3031 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:43.196681 2827 ts_manager.cc:194] Registered new tserver with Master: f346cbc4045b4227a67871a920c7bf86 (127.21.205.193:43249)
I20260925 14:47:43.197726 2827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:47935
I20260925 14:47:43.389205 3167 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:43.389343 3167 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:43.389715 2827 ts_manager.cc:194] Registered new tserver with Master: 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319)
I20260925 14:47:43.390229 2827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:42389
Sep 25 14:47:43 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:43.564119 3302 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:43.564289 3302 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:47:43.564594 2827 ts_manager.cc:194] Registered new tserver with Master: a5c317052f1240b197cde33f3ce50747 (127.21.205.195:45697)
I20260925 14:47:43.565141 2827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:46629
I20260925 14:47:43.565687 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:47:43.574820 3326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.566654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.21.205.254:32805)
0925 14:47:43.566923 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.566925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.566932 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:47:43.566953 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.566954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.566965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.567031 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:43.567743 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.568535 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.569610 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.569955 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.570027 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.571535 (+ 1508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:43.571537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:43.571537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:43.571544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:43.573361 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:43.573773 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:43.573775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:43.573775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:43.573836 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:43.574079 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:43.574080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:43.574080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:43.574209 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:43.574609 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.574649 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.574724 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":61,"threads_started":1}
I20260925 14:47:43.584482 2827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:47:43.586983 2827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:47:43.595659 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.590960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52086 (local address 127.21.205.193:43249)
0925 14:47:43.591098 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.591102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.591346 (+ 244us) server_negotiation.cc:407] Connection header received
0925 14:47:43.591451 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.591453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.591469 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.591520 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.591821 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.592917 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.594393 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.595134 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.595207 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.595269 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.595523 (+ 254us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.595567 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260925 14:47:43.595997 3336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.590370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43249 (local address 127.0.0.1:52086)
0925 14:47:43.591289 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.591299 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.591372 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.591641 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.591644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.591646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.591713 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.591715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.593014 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.593014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.594294 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.594296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.595331 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.595332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.595410 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.595423 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.595851 (+ 428us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.595921 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":904,"mutex_wait_us":7,"thread_start_us":54,"threads_started":1}
I20260925 14:47:43.596181 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.590491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37318 (local address 127.21.205.195:45697)
0925 14:47:43.590620 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.590625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.591408 (+ 783us) server_negotiation.cc:407] Connection header received
0925 14:47:43.591533 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.591536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.591553 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.591609 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.591903 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.592970 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.594525 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.595345 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.595396 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.595443 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.596062 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.596105 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":56,"threads_started":1}
I20260925 14:47:43.596685 3332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.591172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:45697 (local address 127.0.0.1:37318)
0925 14:47:43.591335 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.591346 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.591440 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.591744 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.591745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.591748 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.591817 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.591819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.593312 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.593313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.594426 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.594429 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.595825 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.595875 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.595956 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.595961 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.596568 (+ 607us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.596618 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260925 14:47:43.597007 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 (DEFAULT_TABLE table=test-table [id=5fe7911ae01642cd94e2b4f06466bf54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:43.597347 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb113872e8e44c79b3b68d9eea97240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:43.597510 3235 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 (DEFAULT_TABLE table=test-table [id=5fe7911ae01642cd94e2b4f06466bf54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:43.597855 3235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb113872e8e44c79b3b68d9eea97240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:43.601778 3338 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Bootstrap starting.
I20260925 14:47:43.602464 3338 tablet_bootstrap.cc:654] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:43.602820 3338 log.cc:824] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:43.603626 3338 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: No bootstrap required, opened a new log
I20260925 14:47:43.603724 3338 ts_tablet_manager.cc:1397] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:43.606168 3338 raft_consensus.cc:348] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.606356 3338 raft_consensus.cc:374] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:43.606407 3338 raft_consensus.cc:729] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5c317052f1240b197cde33f3ce50747, State: Initialized, Role: FOLLOWER
I20260925 14:47:43.606523 3338 consensus_queue.cc:260] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.606765 3338 ts_tablet_manager.cc:1428] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:47:43.606894 3302 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
I20260925 14:47:43.608533 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.590168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42319 (local address 127.0.0.1:59670)
0925 14:47:43.590337 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.590379 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.590452 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.604037 (+ 13585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.604038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.604041 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.604089 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.604090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.605591 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.605592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.606703 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.606706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.608254 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.608255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.608314 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.608318 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.608397 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.608451 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260925 14:47:43.608845 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.590727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.21.205.194:42319)
0925 14:47:43.603883 (+ 13156us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.603887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.603897 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:47:43.603924 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.603926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.603939 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.604039 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.604164 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.605487 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.607093 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.608172 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.608243 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.608288 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.608405 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.608442 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13115,"thread_start_us":61,"threads_started":1}
I20260925 14:47:43.609504 3099 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 (DEFAULT_TABLE table=test-table [id=5fe7911ae01642cd94e2b4f06466bf54]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:43.609819 3099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb113872e8e44c79b3b68d9eea97240. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:43.611943 3337 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Bootstrap starting.
I20260925 14:47:43.612612 3337 tablet_bootstrap.cc:654] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:43.612838 3337 log.cc:824] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:43.613257 3341 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Bootstrap starting.
I20260925 14:47:43.614243 3341 tablet_bootstrap.cc:654] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:43.614602 3341 log.cc:824] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:43.615415 3341 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: No bootstrap required, opened a new log
I20260925 14:47:43.615482 3341 ts_tablet_manager.cc:1397] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:43.618117 3341 raft_consensus.cc:348] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.618273 3341 raft_consensus.cc:374] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:43.618314 3341 raft_consensus.cc:729] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a079e5149c044f5838fab7d78510173, State: Initialized, Role: FOLLOWER
I20260925 14:47:43.618431 3341 consensus_queue.cc:260] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.618676 3341 ts_tablet_manager.cc:1428] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:47:43.618906 3167 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
I20260925 14:47:43.620513 3337 tablet_bootstrap.cc:492] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: No bootstrap required, opened a new log
I20260925 14:47:43.620613 3337 ts_tablet_manager.cc:1397] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260925 14:47:43.623090 3337 raft_consensus.cc:348] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
W20260925 14:47:43.623220 3168 tablet.cc:2369] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:47:43.623274 3337 raft_consensus.cc:374] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:43.623322 3337 raft_consensus.cc:729] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f346cbc4045b4227a67871a920c7bf86, State: Initialized, Role: FOLLOWER
I20260925 14:47:43.623436 3337 consensus_queue.cc:260] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.623703 3337 ts_tablet_manager.cc:1428] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:47:43.623869 3031 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
Sep 25 14:47:43 dist-test-slave-d7lw krb5kdc[1814](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347663, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for HTTP/127.21.205.212@KRBTEST.COM
I20260925 14:47:43.662009 3345 raft_consensus.cc:482] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:43.662132 3345 raft_consensus.cc:504] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.662513 3345 leader_election.cc:288] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5c317052f1240b197cde33f3ce50747 (127.21.205.195:45697), 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319)
I20260925 14:47:43.667436 3340 raft_consensus.cc:482] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:43.667548 3340 raft_consensus.cc:504] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.667619 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.663185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:56867 (local address 127.21.205.194:42319)
0925 14:47:43.663228 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.663230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.663504 (+ 274us) server_negotiation.cc:407] Connection header received
0925 14:47:43.663601 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.663603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.663611 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.663639 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.663863 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.664682 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.666597 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.667188 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.667250 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.667259 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.667512 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.667551 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:47:43.667889 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.662919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42319 (local address 127.21.205.193:56867)
0925 14:47:43.663440 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.663450 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.663530 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.663700 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.663701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.663702 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.663768 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.663770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.665134 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.665135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.666500 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.666503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.667361 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.667362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.667432 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.667436 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.667732 (+ 296us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.667772 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":75,"threads_started":1}
I20260925 14:47:43.667929 3340 leader_election.cc:288] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319), f346cbc4045b4227a67871a920c7bf86 (127.21.205.193:43249)
I20260925 14:47:43.668170 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "f346cbc4045b4227a67871a920c7bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a079e5149c044f5838fab7d78510173" is_pre_election: true
I20260925 14:47:43.668308 3116 raft_consensus.cc:2454] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f346cbc4045b4227a67871a920c7bf86 in term 0.
I20260925 14:47:43.668499 2917 leader_election.cc:302] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a079e5149c044f5838fab7d78510173, f346cbc4045b4227a67871a920c7bf86; no voters:
I20260925 14:47:43.668601 3345 raft_consensus.cc:2781] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:47:43.668632 3345 raft_consensus.cc:482] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:47:43.668668 3345 raft_consensus.cc:3037] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:43.669539 3345 raft_consensus.cc:504] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.669653 3345 leader_election.cc:288] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [CANDIDATE]: Term 1 election: Requested vote from peers a5c317052f1240b197cde33f3ce50747 (127.21.205.195:45697), 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319)
I20260925 14:47:43.670051 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "f346cbc4045b4227a67871a920c7bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a079e5149c044f5838fab7d78510173"
I20260925 14:47:43.670125 3116 raft_consensus.cc:3037] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:43.670936 3116 raft_consensus.cc:2454] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f346cbc4045b4227a67871a920c7bf86 in term 1.
I20260925 14:47:43.671123 2917 leader_election.cc:302] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a079e5149c044f5838fab7d78510173, f346cbc4045b4227a67871a920c7bf86; no voters:
I20260925 14:47:43.671249 3345 raft_consensus.cc:2781] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:43.671379 3345 raft_consensus.cc:686] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [term 1 LEADER]: Becoming Leader. State: Replica: f346cbc4045b4227a67871a920c7bf86, State: Running, Role: LEADER
I20260925 14:47:43.671459 3345 consensus_queue.cc:237] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } }
I20260925 14:47:43.672334 2827 catalog_manager.cc:5495] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 reported cstate change: term changed from 0 to 1, leader changed from <none> to f346cbc4045b4227a67871a920c7bf86 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "f346cbc4045b4227a67871a920c7bf86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f346cbc4045b4227a67871a920c7bf86" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43249 } health_report { overall_health: HEALTHY } } }
I20260925 14:47:43.676678 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.668767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33983 (local address 127.21.205.193:43249)
0925 14:47:43.669706 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.669707 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.673281 (+ 3574us) server_negotiation.cc:407] Connection header received
0925 14:47:43.673373 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.673374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.673381 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.673407 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.673578 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.674260 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.675709 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.676296 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.676344 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.676352 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.676581 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.676618 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922}
I20260925 14:47:43.677039 3350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.668415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43249 (local address 127.21.205.195:33983)
0925 14:47:43.673233 (+ 4818us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.673243 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.673313 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.673456 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.673457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.673458 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.673517 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.673518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.674352 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.674353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.675610 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.675613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.676431 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.676432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.676505 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.676509 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.676893 (+ 384us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.676940 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4803,"thread_start_us":59,"threads_started":1}
I20260925 14:47:43.677457 2983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "a5c317052f1240b197cde33f3ce50747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f346cbc4045b4227a67871a920c7bf86" is_pre_election: true
W20260925 14:47:43.677983 3032 tablet.cc:2369] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:47:43.681459 3333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.663132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:49063 (local address 127.21.205.195:45697)
0925 14:47:43.663182 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.663185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.663194 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:47:43.663221 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.663222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.663243 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.663294 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.673275 (+ 9981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.673997 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.680497 (+ 6500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.681060 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.681108 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.681115 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.681339 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.681375 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 14:47:43.681653 3347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.662694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:45697 (local address 127.21.205.193:49063)
0925 14:47:43.662865 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.662877 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.662937 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.663367 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.663368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.663371 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.673139 (+ 9768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.673142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.679242 (+ 6100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.679243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.680384 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.680387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.681173 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.681173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.681240 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.681243 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.681535 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.681577 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":101,"threads_started":1}
I20260925 14:47:43.682022 3255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "f346cbc4045b4227a67871a920c7bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5c317052f1240b197cde33f3ce50747" is_pre_election: true
I20260925 14:47:43.682123 3255 raft_consensus.cc:2454] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f346cbc4045b4227a67871a920c7bf86 in term 0.
I20260925 14:47:43.682205 3334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.668806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:60743 (local address 127.21.205.194:42319)
0925 14:47:43.668839 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.668841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.671916 (+ 3075us) server_negotiation.cc:407] Connection header received
0925 14:47:43.671977 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.671978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.671987 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.672045 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:43.674318 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.675004 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.679216 (+ 4212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.679734 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.679819 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.679827 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.682108 (+ 2281us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.682148 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 14:47:43.682224 3254 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "f346cbc4045b4227a67871a920c7bf86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5c317052f1240b197cde33f3ce50747"
I20260925 14:47:43.682264 3254 raft_consensus.cc:3037] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:43.682412 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.668227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:42319 (local address 127.21.205.195:60743)
0925 14:47:43.671849 (+ 3622us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.671861 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:43.671929 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:43.672982 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:43.672983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:43.672985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:43.673062 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.673064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.678034 (+ 4970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.678035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.679141 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:43.679144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.681941 (+ 2797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.681942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:43.682014 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.682019 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:47:43.682296 (+ 277us) client_negotiation.cc:240] Negotiation successful
0925 14:47:43.682337 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3604,"thread_start_us":85,"threads_started":1}
I20260925 14:47:43.682654 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb113872e8e44c79b3b68d9eea97240" candidate_uuid: "a5c317052f1240b197cde33f3ce50747" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a079e5149c044f5838fab7d78510173" is_pre_election: true
I20260925 14:47:43.682760 3116 raft_consensus.cc:2379] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5c317052f1240b197cde33f3ce50747 in current term 1: Already voted for candidate f346cbc4045b4227a67871a920c7bf86 in this term.
I20260925 14:47:43.683158 3254 raft_consensus.cc:2454] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f346cbc4045b4227a67871a920c7bf86 in term 1.
I20260925 14:47:43.683236 3190 leader_election.cc:302] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5c317052f1240b197cde33f3ce50747; no voters: 5a079e5149c044f5838fab7d78510173, f346cbc4045b4227a67871a920c7bf86
I20260925 14:47:43.684053 3340 raft_consensus.cc:2726] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 14:47:43.688975 3335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.685689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52092 (local address 127.21.205.193:43249)
0925 14:47:43.685723 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:43.685725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:43.685730 (+ 5us) server_negotiation.cc:407] Connection header received
0925 14:47:43.685757 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:43.685758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:43.685774 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:43.685828 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:47:43.686012 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.686729 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.687847 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:43.688121 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:43.688185 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:43.688379 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:47:43.688537 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:47:43.688644 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:43.688893 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 14:47:43.688912 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:47:43.691008 3116 raft_consensus.cc:1261] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 1 FOLLOWER]: Refusing update from remote peer f346cbc4045b4227a67871a920c7bf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:43.691308 3345 consensus_queue.cc:1035] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a079e5149c044f5838fab7d78510173" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 42319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:43.693825 3254 raft_consensus.cc:1261] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 1 FOLLOWER]: Refusing update from remote peer f346cbc4045b4227a67871a920c7bf86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:43.694149 3345 consensus_queue.cc:1035] T 7cb113872e8e44c79b3b68d9eea97240 P f346cbc4045b4227a67871a920c7bf86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5c317052f1240b197cde33f3ce50747" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 45697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:43.699165 3354 mvcc.cc:205] Tried to move back new op lower bound from 7333264030475673600 to 7333264030398746624. Current Snapshot: MvccSnapshot[applied={T|T < 7333264030475673600}]
I20260925 14:47:43.700865 3355 mvcc.cc:205] Tried to move back new op lower bound from 7333264030475673600 to 7333264030398746624. Current Snapshot: MvccSnapshot[applied={T|T < 7333264030475673600}]
I20260925 14:47:43.701010 3357 mvcc.cc:205] Tried to move back new op lower bound from 7333264030475673600 to 7333264030398746624. Current Snapshot: MvccSnapshot[applied={T|T < 7333264030475673600}]
I20260925 14:47:43.703949 2827 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:43.704036 2827 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:43.704769 2827 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 125dbb071a664166af56580eecbe16f8: Sending DeleteTablet for 3 replicas of tablet 7cb113872e8e44c79b3b68d9eea97240
I20260925 14:47:43.705350 3235 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:43 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:37318
I20260925 14:47:43.705564 3099 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:43 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:59670
I20260925 14:47:43.705751 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb113872e8e44c79b3b68d9eea97240 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:43 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:52086
I20260925 14:47:43.706538 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2867
I20260925 14:47:43.708844 3364 tablet_replica.cc:330] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: stopping tablet replica
I20260925 14:47:43.708971 3363 tablet_replica.cc:330] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: stopping tablet replica
I20260925 14:47:43.709110 3364 raft_consensus.cc:2229] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:43.709215 3363 raft_consensus.cc:2229] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:43.709343 3363 raft_consensus.cc:2258] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:43.709486 3364 raft_consensus.cc:2258] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:43.709973 3364 ts_tablet_manager.cc:1905] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:43.710062 3363 ts_tablet_manager.cc:1905] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:43.711704 3363 ts_tablet_manager.cc:1918] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:43.711848 3363 log.cc:1196] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/7cb113872e8e44c79b3b68d9eea97240
I20260925 14:47:43.712170 3363 ts_tablet_manager.cc:1939] T 7cb113872e8e44c79b3b68d9eea97240 P a5c317052f1240b197cde33f3ce50747: Deleting consensus metadata
I20260925 14:47:43.712664 2812 catalog_manager.cc:4841] TS a5c317052f1240b197cde33f3ce50747 (127.21.205.195:45697): tablet 7cb113872e8e44c79b3b68d9eea97240 (table test-table [id=5fe7911ae01642cd94e2b4f06466bf54]) successfully deleted
I20260925 14:47:43.714924 3364 ts_tablet_manager.cc:1918] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:43.715022 3364 log.cc:1196] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/7cb113872e8e44c79b3b68d9eea97240
I20260925 14:47:43.715278 3364 ts_tablet_manager.cc:1939] T 7cb113872e8e44c79b3b68d9eea97240 P 5a079e5149c044f5838fab7d78510173: Deleting consensus metadata
I20260925 14:47:43.715668 2813 catalog_manager.cc:4841] TS 5a079e5149c044f5838fab7d78510173 (127.21.205.194:42319): tablet 7cb113872e8e44c79b3b68d9eea97240 (table test-table [id=5fe7911ae01642cd94e2b4f06466bf54]) successfully deleted
W20260925 14:47:43.715865 2813 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:43249 (error 108)
I20260925 14:47:43.716071 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3035
I20260925 14:47:43.720624 3331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:43.720493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43249 (local address 127.0.0.1:52096)
0925 14:47:43.720551 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:43.720588 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 14:47:43.720835 2813 catalog_manager.cc:4584] TS f346cbc4045b4227a67871a920c7bf86 (127.21.205.193:43249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cb113872e8e44c79b3b68d9eea97240: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43249: connect: Connection refused (error 111)
I20260925 14:47:43.725276 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3170
I20260925 14:47:43.732486 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 2794
I20260925 14:47:43.743654 22327 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 14:47:44.088445 22327 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 14:47:44.088555 22327 mini_ranger.cc:66] Stopping Ranger...
I20260925 14:47:44.425384 22327 mini_ranger.cc:68] Stopped Ranger
2026-09-25 14:47:44.425 UTC [1845] LOG: received smart shutdown request
2026-09-25 14:47:44.429 UTC [1845] LOG: background worker "logical replication launcher" (PID 1854) exited with exit code 1
2026-09-25 14:47:44.429 UTC [1849] LOG: shutting down
2026-09-25 14:47:44.586 UTC [1845] LOG: database system is shut down
2026-09-25T14:47:44Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (45334 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:47:44 dist-test-slave-d7lw krb5kdc[3376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:47:44 dist-test-slave-d7lw krb5kdc[3376](info): set up 2 sockets
Sep 25 14:47:44 dist-test-slave-d7lw krb5kdc[3376](info): commencing operation
krb5kdc: starting...
W20260925 14:47:45.722385 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.003s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:47:45 dist-test-slave-d7lw krb5kdc[3376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:47:45.834928 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35895
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:45.914238 3390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:45.914463 3390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:45.914498 3390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:45.915649 3390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:45.915716 3390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:45.915740 3390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:45.915762 3390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:45.915818 3390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:45.917263 3390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:35895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:45.917529 3390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:45.917753 3390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:45.922520 3396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:45.922529 3398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:45.922587 3395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:45.922785 3390 server_base.cc:1034] running on GCE node
I20260925 14:47:45.923029 3390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:45.923321 3390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:45.924485 3390 hybrid_clock.cc:648] HybridClock initialized: now 1790347665924474 us; error 39 us; skew 500 ppm
Sep 25 14:47:45 dist-test-slave-d7lw krb5kdc[3376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:45.931679 3390 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:45.932171 3390 webserver.cc:460] Webserver started at http://127.21.205.254:41581/ using document root <none> and password file <none>
I20260925 14:47:45.932413 3390 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:45.932498 3390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:45.932651 3390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:45.933671 3390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "d35a81cbb1a347cb868ae01e0cbcfa26"
format_stamp: "Formatted at 2026-09-25 14:47:45 on dist-test-slave-d7lw"
server_key: "cc587f91e86f22611d51c7341bb8084b"
server_key_iv: "332eca1decd26eae60179c8b7a431f7a"
server_key_version: "encryptionkey@0"
I20260925 14:47:45.934027 3390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "d35a81cbb1a347cb868ae01e0cbcfa26"
format_stamp: "Formatted at 2026-09-25 14:47:45 on dist-test-slave-d7lw"
server_key: "cc587f91e86f22611d51c7341bb8084b"
server_key_iv: "332eca1decd26eae60179c8b7a431f7a"
server_key_version: "encryptionkey@0"
I20260925 14:47:45.935259 3390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:47:45.935962 3405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:45.936128 3390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:47:45.936244 3390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "d35a81cbb1a347cb868ae01e0cbcfa26"
format_stamp: "Formatted at 2026-09-25 14:47:45 on dist-test-slave-d7lw"
server_key: "cc587f91e86f22611d51c7341bb8084b"
server_key_iv: "332eca1decd26eae60179c8b7a431f7a"
server_key_version: "encryptionkey@0"
I20260925 14:47:45.936323 3390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:45.944070 3390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:45.945725 3390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:45.945873 3390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:45.950001 3390 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:35895
I20260925 14:47:45.950060 3458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:35895 every 8 connection(s)
I20260925 14:47:45.950397 3390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:45.951159 3459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:45.952744 3459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:47:45 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:45.958922 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3390
I20260925 14:47:45.959028 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:47:45.959231 22327 external_mini_cluster.cc:1428] Setting key e67255bbc245084b377bed1e31922261
I20260925 14:47:45.961628 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:45.953338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35895 (local address 127.0.0.1:33274)
0925 14:47:45.953499 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:45.953535 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:45.953978 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:45.954220 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:45.954222 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:45.954452 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:45.954596 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:45.954600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:45.955524 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:45.955524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:45.956275 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:45.956277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:45.956723 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:45.956724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:45.956778 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:45.957171 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:45.957178 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:45.958451 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:45.960223 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:45.960229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:45.960231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:45.960504 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:45.960774 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:45.960775 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:45.960775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:45.960851 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:45.961255 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:45.961258 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:45.961307 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 14:47:45.961379 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:47:45.961489 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260925 14:47:45.961725 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:45.953378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33274 (local address 127.21.205.254:35895)
0925 14:47:45.953535 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:45.953542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:45.953616 (+ 74us) server_negotiation.cc:407] Connection header received
0925 14:47:45.954097 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:45.954098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:45.954113 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:45.954191 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:45.954672 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:45.955445 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:45.956344 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:45.956650 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:45.956726 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:45.958527 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:45.958533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:45.958534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:45.958539 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:45.960146 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:45.960593 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:45.960596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:45.960596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:45.960704 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:45.960924 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:45.960925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:45.960925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:45.961187 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:45.961268 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:45.961426 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 14:47:45.961576 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260925 14:47:45.962322 3459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:45.962653 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:45.962540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42212)
0925 14:47:45.962583 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:45.962626 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 14:47:45.962884 3408 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
I20260925 14:47:45.963311 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:45.963206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42214)
0925 14:47:45.963251 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:45.963281 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 14:47:45.963419 3459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33377: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.205.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
I20260925 14:47:45.982362 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:33377
--webserver_interface=127.21.205.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:46.012637 3459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } attempt: 1
I20260925 14:47:46.013170 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.013026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42228)
0925 14:47:46.013095 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.013138 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260925 14:47:46.013522 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.013434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42240)
0925 14:47:46.013476 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.013502 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 14:47:46.013641 3459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33377: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
W20260925 14:47:46.060789 3467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:46.060989 3467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:46.061010 3467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:46.062150 3467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:46.062189 3467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:46.062202 3467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:46.062214 3467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:46.062268 3467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:46.063675 3467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:33377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.205.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:46.063951 3467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:46.064180 3467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:47:46.068867 3467 server_base.cc:1034] running on GCE node
W20260925 14:47:46.068877 3473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:46.068877 3472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:46.068995 3475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:46.069324 3467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:46.069566 3467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:46.070701 3467 hybrid_clock.cc:648] HybridClock initialized: now 1790347666070679 us; error 38 us; skew 500 ppm
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:46.073113 3467 init.cc:377] Logged in from keytab as kudu/127.21.205.253@KRBTEST.COM (short username kudu)
I20260925 14:47:46.073525 3467 webserver.cc:460] Webserver started at http://127.21.205.253:34613/ using document root <none> and password file <none>
I20260925 14:47:46.073748 3467 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:46.073813 3467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:46.073957 3467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:46.074918 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data/instance:
uuid: "498ab4addf3149d6a830cb4c5744ca93"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "76297454cd7fbe14649c7fbf93e811fb"
server_key_iv: "6c51080594203ba9180094286a5eef0e"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.075280 3467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal/instance:
uuid: "498ab4addf3149d6a830cb4c5744ca93"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "76297454cd7fbe14649c7fbf93e811fb"
server_key_iv: "6c51080594203ba9180094286a5eef0e"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.076668 3467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:47:46.077358 3482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:46.077545 3467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:46.077641 3467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "498ab4addf3149d6a830cb4c5744ca93"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "76297454cd7fbe14649c7fbf93e811fb"
server_key_iv: "6c51080594203ba9180094286a5eef0e"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.077715 3467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:46.081815 3459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } attempt: 2
I20260925 14:47:46.082155 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.082042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42250)
0925 14:47:46.082094 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.082127 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:47:46.082492 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.082412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42262)
0925 14:47:46.082452 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.082476 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 14:47:46.082590 3459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33377: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33377: connect: Connection refused (error 111)
I20260925 14:47:46.090691 3467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:46.092839 3467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:46.092994 3467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:46.096890 3467 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.253:33377
I20260925 14:47:46.096928 3534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.253:33377 every 8 connection(s)
I20260925 14:47:46.097244 3467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
I20260925 14:47:46.098104 3535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:46.099506 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:46.106418 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3467
I20260925 14:47:46.106524 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-1/wal/instance
I20260925 14:47:46.106712 22327 external_mini_cluster.cc:1428] Setting key 5c035e7ee755943e4eb65595b9c23bd1
I20260925 14:47:46.107903 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.100194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.21.205.254:35895)
0925 14:47:46.100252 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.100254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.100443 (+ 189us) server_negotiation.cc:407] Connection header received
0925 14:47:46.100909 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.100911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.100927 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.100982 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.101542 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.102039 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.103029 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.103233 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.103308 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.105183 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.105184 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.105185 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.105192 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.106484 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.106875 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.106876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.106876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.106944 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.107179 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.107180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.107180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.107326 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.107394 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.107626 (+ 232us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.107717 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260925 14:47:46.107916 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.100179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35895 (local address 127.0.0.1:33284)
0925 14:47:46.100332 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.100362 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.100800 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.101051 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.101054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.101269 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.101450 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.101454 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.102134 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.102135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.102916 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.102918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.103314 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.103315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.103438 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.103872 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.103880 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.105093 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.106575 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.106587 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.106588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.106797 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.107012 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.107013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.107013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.107111 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.107388 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.107393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.107510 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.107585 (+ 75us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.107699 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260925 14:47:46.108533 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.205.252@KRBTEST.COM; defaulting to no policy
I20260925 14:47:46.115773 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.108759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42264)
0925 14:47:46.108807 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.108818 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.108855 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.109094 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.109095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.109242 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.109298 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.109300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.110075 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.110077 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.110949 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.110952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.111372 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.111373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.111467 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.111794 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.111799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.113042 (+ 1243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.114679 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.114680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.114681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.114882 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.115115 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.115115 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.115116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.115167 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.115462 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.115463 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.115500 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.115564 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.115660 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 14:47:46.115866 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.108813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.21.205.253:33377)
0925 14:47:46.108966 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.108969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.108982 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:47:46.108999 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.109001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.109010 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.109071 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.109365 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.109989 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.111035 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.111302 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.111348 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.113137 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.113145 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.113145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.113152 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.114581 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.114983 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.114986 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.114986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.115040 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.115234 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.115235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.115235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.115373 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.115477 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.115594 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.115711 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":119,"thread_start_us":94,"threads_started":1}
I20260925 14:47:46.116441 3535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:46.116755 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.116644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50718)
0925 14:47:46.116689 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.116733 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 14:47:46.116891 3484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
I20260925 14:47:46.117251 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.117160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50726)
0925 14:47:46.117208 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.117231 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.252@KRBTEST.COM" created.
W20260925 14:47:46.117369 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:41073: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
I20260925 14:47:46.130944 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:41073
--webserver_interface=127.21.205.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:40489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:46.166617 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } attempt: 1
I20260925 14:47:46.167150 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.167013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50732)
0925 14:47:46.167080 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.167120 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 14:47:46.167524 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.167441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50748)
0925 14:47:46.167485 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.167508 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 14:47:46.167645 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:41073: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
I20260925 14:47:46.173730 3459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } attempt: 3
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:47:46.180469 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.174090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.21.205.253:33377)
0925 14:47:46.174145 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.174147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.174158 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:47:46.174217 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.174218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.174230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.174296 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.174645 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.175169 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.175887 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.176121 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.176212 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.178015 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.178017 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.178017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.178024 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.179337 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.179659 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.179661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.179661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.179717 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.179946 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.179947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.179947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.180093 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.180155 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.180299 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.180369 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:47:46.180467 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.174041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42268)
0925 14:47:46.174080 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.174089 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.174158 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.174316 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.174317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.174502 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.174571 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.174573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.175269 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.175269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.175773 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.175775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.176212 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.176213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.176293 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.176758 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.176762 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.177928 (+ 1166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.179423 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.179424 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.179425 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.179588 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.179817 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.179817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.179818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.179880 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.180146 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.180147 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.180190 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.180276 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.180368 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260925 14:47:46.181018 3459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:46.181358 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.181285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50762)
0925 14:47:46.181318 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.181343 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260925 14:47:46.181712 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.181647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50764)
0925 14:47:46.181681 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.181699 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260925 14:47:46.181809 3459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:41073: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
W20260925 14:47:46.209548 3542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:46.209712 3542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:46.209731 3542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:46.210765 3542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:46.210803 3542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:46.210817 3542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:46.210829 3542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:46.210877 3542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:46.212342 3542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:35895,127.21.205.253:33377,127.21.205.252:41073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:41073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.205.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:46.212615 3542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:46.212834 3542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:47:46.212946 3459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } attempt: 1
I20260925 14:47:46.213325 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.213206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50778)
0925 14:47:46.213261 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.213292 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 14:47:46.213721 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.213646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50790)
0925 14:47:46.213683 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.213706 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 14:47:46.213852 3459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:41073: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
W20260925 14:47:46.217716 3550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:46.217794 3548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:46.217722 3547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:46.218089 3542 server_base.cc:1034] running on GCE node
I20260925 14:47:46.218289 3542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:46.218493 3542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:46.219623 3542 hybrid_clock.cc:648] HybridClock initialized: now 1790347666219623 us; error 33 us; skew 500 ppm
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:46.221879 3542 init.cc:377] Logged in from keytab as kudu/127.21.205.252@KRBTEST.COM (short username kudu)
I20260925 14:47:46.222225 3542 webserver.cc:460] Webserver started at http://127.21.205.252:36191/ using document root <none> and password file <none>
I20260925 14:47:46.222390 3542 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:46.222430 3542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:46.222510 3542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:46.223419 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data/instance:
uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "1265c64243f9b58abfeaa5d71a4f8ff1"
server_key_iv: "c6d82b34087545993fb63396afd60513"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.223692 3542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal/instance:
uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "1265c64243f9b58abfeaa5d71a4f8ff1"
server_key_iv: "c6d82b34087545993fb63396afd60513"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.224884 3542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:46.225546 3557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:46.225708 3542 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:47:46.225798 3542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3"
format_stamp: "Formatted at 2026-09-25 14:47:46 on dist-test-slave-d7lw"
server_key: "1265c64243f9b58abfeaa5d71a4f8ff1"
server_key_iv: "c6d82b34087545993fb63396afd60513"
server_key_version: "encryptionkey@0"
I20260925 14:47:46.225872 3542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:46.235796 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } attempt: 2
I20260925 14:47:46.236145 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.236036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50794)
0925 14:47:46.236086 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.236119 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:47:46.236490 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.236413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50800)
0925 14:47:46.236452 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.236474 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 14:47:46.236603 3535 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:41073: Network error: Client connection negotiation failed: client connection to 127.21.205.252:41073: connect: Connection refused (error 111)
I20260925 14:47:46.245548 3542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:46.247238 3542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:46.247402 3542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:46.251235 3542 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.252:41073
I20260925 14:47:46.251299 3610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.252:41073 every 8 connection(s)
I20260925 14:47:46.251608 3542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
I20260925 14:47:46.252532 3611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:46.253976 3611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:46.255648 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3542
I20260925 14:47:46.255762 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790347486435597-22327-0/minicluster-data/master-2/wal/instance
I20260925 14:47:46.255957 22327 external_mini_cluster.cc:1428] Setting key 384fec6869d39fa095c08ffd3065a5db
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:47:46.264499 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.254643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.21.205.254:35895)
0925 14:47:46.254715 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.254718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.254917 (+ 199us) server_negotiation.cc:407] Connection header received
0925 14:47:46.255307 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.255308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.255317 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.255382 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.256060 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.256833 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.258278 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.258480 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.258571 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.261013 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.261014 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.261015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.261021 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.263110 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.263537 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.263538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.263538 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.263582 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.263861 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.263862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.263862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.263981 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.264047 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.264298 (+ 251us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.264393 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 14:47:46.264616 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.254621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:35895 (local address 127.0.0.1:33296)
0925 14:47:46.254798 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.254835 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.255235 (+ 400us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.255429 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.255431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.255661 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.255957 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.255962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.256949 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.256950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.258202 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.258204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.258582 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.258582 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.258681 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.259154 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.259161 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.260930 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.263182 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.263189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.263191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.263468 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.263644 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.263645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.263645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.263761 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.264051 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.264056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.264150 (+ 94us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.264247 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.264385 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260925 14:47:46.265134 3611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:46.265461 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.256311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.21.205.252:41073)
0925 14:47:46.256455 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.256459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.256507 (+ 48us) server_negotiation.cc:407] Connection header received
0925 14:47:46.256550 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.256552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.256571 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.256640 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.257452 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.258404 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.259357 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.259825 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.259899 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.262226 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.262232 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.262232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.262238 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.263927 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.264237 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.264240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.264240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.264292 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.264486 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.264487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.264487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.264646 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.264818 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.265073 (+ 255us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.265144 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:46.271694 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.265448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.21.205.253:33377)
0925 14:47:46.265494 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.265496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.265505 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:47:46.265554 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.265555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.265565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.265635 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.265913 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.266652 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.267263 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.267478 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.267564 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.269325 (+ 1761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.269326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.269327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.269334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.270609 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.270977 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.270977 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.270978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.271026 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.271222 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.271223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.271223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.271356 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.271426 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.271569 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.271626 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:47:46.271727 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.265382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33377 (local address 127.0.0.1:42276)
0925 14:47:46.265439 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.265447 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.265484 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.265636 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.265636 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.265759 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.265840 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.265842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.266724 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.266725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.267176 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.267177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.267561 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.267562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.267628 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.268007 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.268011 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.269228 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.270704 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.270705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.270705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.270903 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.271093 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.271094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.271094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.271158 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.271423 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.271424 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.271469 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.271552 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.271633 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 14:47:46.272125 3611 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:47:46.273242 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.266125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33298 (local address 127.21.205.254:35895)
0925 14:47:46.266153 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.266155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.266163 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:47:46.266183 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.266184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.266191 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.266252 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.266783 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.267224 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.267898 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.268084 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.268158 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.270467 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.270468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.270469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.270475 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.272141 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.272529 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.272530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.272530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.272588 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.272785 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.272786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.272786 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.272926 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.273049 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.273127 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.273186 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13}
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:47:46.279078 3613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.272374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50826)
0925 14:47:46.272423 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.272433 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.272478 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.272609 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.272610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.272735 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.272786 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.272787 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.273622 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.273623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.274184 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.274186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.274599 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.274600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.274649 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.274975 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.274979 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.276297 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.278080 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.278081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.278081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.278252 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.278462 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.278463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.278463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.278513 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.278763 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.278764 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.278798 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.278850 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.278957 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 14:47:46.279178 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.272426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50826 (local address 127.21.205.252:41073)
0925 14:47:46.272471 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.272473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.272481 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:47:46.272546 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.272547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.272556 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.272610 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.272857 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.273538 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.274255 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.274538 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.274581 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.276372 (+ 1791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.276373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.276373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.276378 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.278000 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.278336 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.278337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.278338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.278396 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.278574 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.278574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.278575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.278688 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.278753 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.278897 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.279009 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":17}
I20260925 14:47:46.280782 3611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: Bootstrap starting.
I20260925 14:47:46.281541 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.273829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42284 (local address 127.21.205.253:33377)
0925 14:47:46.273859 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.273861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.273890 (+ 29us) server_negotiation.cc:407] Connection header received
0925 14:47:46.273925 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.273926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.273933 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.273987 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.274456 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.275052 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.275930 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.276167 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.276220 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.278682 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.278683 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.278683 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.278690 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.280475 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.280837 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.280838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.280838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.280900 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.281097 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.281097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.281097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.281187 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.281309 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.281436 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.281487 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 14:47:46.281798 3611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:46.282068 3611 log.cc:824] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:46.282872 3611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: No bootstrap required, opened a new log
I20260925 14:47:46.283027 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:47:46.284727 3611 raft_consensus.cc:348] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.284991 3611 raft_consensus.cc:374] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:46.285061 3611 raft_consensus.cc:729] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d916d2cb0a5148a4a1c1f1109a96fbc3, State: Initialized, Role: FOLLOWER
I20260925 14:47:46.285265 3611 consensus_queue.cc:260] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.285753 3611 sys_catalog.cc:564] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:46.288177 3624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.288290 3624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:47:46.289011 3459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } attempt: 2
I20260925 14:47:46.287803 3542 master_runner.cc:186] Error getting master registration for 127.21.205.254:35895: OK, instance_id { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" instance_seqno: 1790347665948814 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:47:46.290607 3542 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 14:47:46.290973 3635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:47:46.291070 3635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:47:46.293151 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.284390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50838 (local address 127.21.205.252:41073)
0925 14:47:46.284581 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.284583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.284637 (+ 54us) server_negotiation.cc:407] Connection header received
0925 14:47:46.284735 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.284736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.284746 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.284776 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.285503 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.286191 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287344 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.287649 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287681 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.290079 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.290080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.290080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.290087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.291765 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.292219 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.292220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.292221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.292271 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.292521 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.292522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.292522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.292670 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.292739 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.292973 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.293046 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172}
I20260925 14:47:46.293196 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.284616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.21.205.253:33377)
0925 14:47:46.284646 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.284648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.284656 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:47:46.284678 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.284678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.284687 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.284739 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.285479 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.286105 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287301 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.287581 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287617 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.289180 (+ 1563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.289181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.289181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.289188 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.291490 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.291888 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.291889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.291889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.291939 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.292131 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.292132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.292132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.292242 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.292806 (+ 564us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.293042 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.293116 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260925 14:47:46.293329 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.284391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.21.205.254:35895)
0925 14:47:46.284428 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.284430 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.284756 (+ 326us) server_negotiation.cc:407] Connection header received
0925 14:47:46.284862 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.284863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.284885 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.284914 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.285630 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.286304 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287344 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.287641 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.287678 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.290106 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.290107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.290107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.290113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.291944 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.292317 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.292318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.292318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.292369 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.292590 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.292591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.292591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.292703 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.292784 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.293161 (+ 377us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.293236 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347665, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:47:46.299338 3461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.289208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50854)
0925 14:47:46.289431 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.289441 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.289481 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.290493 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.290494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.290656 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.290692 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.290694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.294086 (+ 3392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.294088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.294581 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.294583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.294971 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.294972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.295022 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.295485 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.295489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.296835 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.298489 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.298490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.298490 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.298644 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.298838 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.298839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.298839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.298889 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.299137 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.299138 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.299166 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.299210 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.299266 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260925 14:47:46.299429 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.289576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.21.205.252:41073)
0925 14:47:46.290349 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.290350 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.290357 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:47:46.290374 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.290374 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.290401 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.291058 (+ 657us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.291120 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.294019 (+ 2899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.294677 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.294904 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.294972 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.296919 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.296921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.296921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.296928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.298407 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.298711 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.298711 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.298712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.298771 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.298949 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.298950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.298950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.299076 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.299151 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.299240 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.299330 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":72,"threads_started":1}
I20260925 14:47:46.301041 3459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: Bootstrap starting.
I20260925 14:47:46.301715 3459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:46.301981 3459 log.cc:824] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:46.302794 3459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: No bootstrap required, opened a new log
I20260925 14:47:46.304720 3459 raft_consensus.cc:348] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.304884 3459 raft_consensus.cc:374] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:46.304920 3459 raft_consensus.cc:729] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d35a81cbb1a347cb868ae01e0cbcfa26, State: Initialized, Role: FOLLOWER
I20260925 14:47:46.305061 3459 consensus_queue.cc:260] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.305359 3459 sys_catalog.cc:564] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:46.305326 3639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.305438 3639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 14:47:46.307159 3650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:47:46.307256 3650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:47:46.307824 3390 master_runner.cc:186] Error getting master registration for 127.21.205.253:33377: OK, instance_id { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" instance_seqno: 1790347666095672 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:47:46.310346 3390 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 14:47:46.327814 3535 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } attempt: 3
Sep 25 14:47:46 dist-test-slave-d7lw krb5kdc[3376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347666, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:47:46.336287 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.328091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:41073 (local address 127.0.0.1:50868)
0925 14:47:46.328135 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:46.328145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:46.328183 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:46.328338 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:46.328339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:46.328514 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:46.328563 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.328564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.329577 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.329578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.330361 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:46.330364 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.330849 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.330850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:46.330912 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.331389 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:46.331394 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:46.332911 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:46.335157 (+ 2246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.335158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.335159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.335379 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.335598 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:46.335599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:46.335599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:46.335663 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:46.335952 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:46.335953 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:46.335998 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 14:47:46.336070 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:47:46.336169 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260925 14:47:46.336295 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:46.328159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50868 (local address 127.21.205.252:41073)
0925 14:47:46.328196 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:46.328198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:46.328206 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:47:46.328231 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:46.328232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:46.328241 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:46.328303 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:46.328650 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.329494 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.330447 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:46.330771 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:46.330838 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:46.332999 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:46.333000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:46.333000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:46.333008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:46.335083 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.335465 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.335466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.335466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.335523 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:46.335728 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:46.335729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:46.335729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:46.335889 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:46.335953 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:46.336107 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 14:47:46.336184 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:47:46.338068 3535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: Bootstrap starting.
I20260925 14:47:46.338841 3535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:46.339183 3535 log.cc:824] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:46.340036 3535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: No bootstrap required, opened a new log
I20260925 14:47:46.342098 3535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.342217 3535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:46.342248 3535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 498ab4addf3149d6a830cb4c5744ca93, State: Initialized, Role: FOLLOWER
I20260925 14:47:46.342360 3535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.342691 3535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:46.342661 3653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.342847 3653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 14:47:46.343530 3488 tablet.cc:2369] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:47:46.345449 3664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:47:46.345554 3664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:47:46.347404 3467 master_runner.cc:263] No leader master found from master 498ab4addf3149d6a830cb4c5744ca93
I20260925 14:47:46.347448 3467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 14:47:46.379840 3653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:46.379969 3653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.380234 3653 leader_election.cc:288] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d35a81cbb1a347cb868ae01e0cbcfa26 (127.21.205.254:35895), d916d2cb0a5148a4a1c1f1109a96fbc3 (127.21.205.252:41073)
I20260925 14:47:46.380720 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498ab4addf3149d6a830cb4c5744ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" is_pre_election: true
I20260925 14:47:46.380901 3584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ab4addf3149d6a830cb4c5744ca93 in term 0.
I20260925 14:47:46.381207 3432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498ab4addf3149d6a830cb4c5744ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" is_pre_election: true
I20260925 14:47:46.381338 3432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 498ab4addf3149d6a830cb4c5744ca93 in term 0.
I20260925 14:47:46.381738 3484 leader_election.cc:302] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ab4addf3149d6a830cb4c5744ca93, d916d2cb0a5148a4a1c1f1109a96fbc3; no voters:
I20260925 14:47:46.381978 3653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:47:46.382030 3653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:47:46.382053 3653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:46.383031 3653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.383165 3653 leader_election.cc:288] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [CANDIDATE]: Term 1 election: Requested vote from peers d35a81cbb1a347cb868ae01e0cbcfa26 (127.21.205.254:35895), d916d2cb0a5148a4a1c1f1109a96fbc3 (127.21.205.252:41073)
I20260925 14:47:46.383572 3432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498ab4addf3149d6a830cb4c5744ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26"
I20260925 14:47:46.383652 3432 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:46.383888 3584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "498ab4addf3149d6a830cb4c5744ca93" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3"
I20260925 14:47:46.383965 3584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:46.384868 3584 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ab4addf3149d6a830cb4c5744ca93 in term 1.
I20260925 14:47:46.385075 3484 leader_election.cc:302] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 498ab4addf3149d6a830cb4c5744ca93, d916d2cb0a5148a4a1c1f1109a96fbc3; no voters:
I20260925 14:47:46.385891 3432 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 498ab4addf3149d6a830cb4c5744ca93 in term 1.
I20260925 14:47:46.386138 3653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:46.386327 3653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [term 1 LEADER]: Becoming Leader. State: Replica: 498ab4addf3149d6a830cb4c5744ca93, State: Running, Role: LEADER
I20260925 14:47:46.386430 3653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } }
I20260925 14:47:46.386790 3653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 498ab4addf3149d6a830cb4c5744ca93. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.386859 3653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:46.387358 3668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:46.387619 3668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:46.389686 3584 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [term 1 FOLLOWER]: Refusing update from remote peer 498ab4addf3149d6a830cb4c5744ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:46.390048 3653 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:46.392513 3624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 498ab4addf3149d6a830cb4c5744ca93. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.392599 3624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:47:46.393024 3432 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [term 1 FOLLOWER]: Refusing update from remote peer 498ab4addf3149d6a830cb4c5744ca93: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:46.393329 3653 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:46.394546 3639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 498ab4addf3149d6a830cb4c5744ca93. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.394629 3639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:47:46.395860 3666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.395948 3666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:46.396466 3639 sys_catalog.cc:455] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.396541 3639 sys_catalog.cc:458] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:47:46.396772 3669 mvcc.cc:205] Tried to move back new op lower bound from 7333264041529675776 to 7333264041519263744. Current Snapshot: MvccSnapshot[applied={T|T < 7333264041529675776}]
I20260925 14:47:46.396994 3672 mvcc.cc:205] Tried to move back new op lower bound from 7333264041529675776 to 7333264041519263744. Current Snapshot: MvccSnapshot[applied={T|T < 7333264041529675776}]
I20260925 14:47:46.397748 3653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.397821 3653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:46.398162 3624 sys_catalog.cc:455] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "498ab4addf3149d6a830cb4c5744ca93" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d35a81cbb1a347cb868ae01e0cbcfa26" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 35895 } } peers { permanent_uuid: "498ab4addf3149d6a830cb4c5744ca93" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33377 } } peers { permanent_uuid: "d916d2cb0a5148a4a1c1f1109a96fbc3" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 41073 } } }
I20260925 14:47:46.398236 3624 sys_catalog.cc:458] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:47:46.398867 3670 mvcc.cc:205] Tried to move back new op lower bound from 7333264041529675776 to 7333264041519263744. Current Snapshot: MvccSnapshot[applied={T|T < 7333264041529675776}]
I20260925 14:47:46.399678 3668 catalog_manager.cc:1347] Generated new cluster ID: 4eca8de4aa724d4b9ad98d38d569983a
I20260925 14:47:46.399724 3668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:47:46.418275 3668 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:47:46.419092 3668 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:46.434119 3668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 498ab4addf3149d6a830cb4c5744ca93: Generated new TSK 0
I20260925 14:47:46.434356 3668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:47:47.291656 3635 catalog_manager.cc:1259] Loaded cluster ID: 4eca8de4aa724d4b9ad98d38d569983a
I20260925 14:47:47.291750 3635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: loading cluster ID for follower catalog manager: success
I20260925 14:47:47.292618 3635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: acquiring CA information for follower catalog manager: success
I20260925 14:47:47.292919 3635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d916d2cb0a5148a4a1c1f1109a96fbc3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:47:47.307677 3650 catalog_manager.cc:1259] Loaded cluster ID: 4eca8de4aa724d4b9ad98d38d569983a
I20260925 14:47:47.307732 3650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: loading cluster ID for follower catalog manager: success
I20260925 14:47:47.308553 3650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: acquiring CA information for follower catalog manager: success
I20260925 14:47:47.308840 3650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d35a81cbb1a347cb868ae01e0cbcfa26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:47:47.489280 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3390
I20260925 14:47:47.494647 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3467
I20260925 14:47:47.499658 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3542
2026-09-25T14:47:47Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2801 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:47:47 dist-test-slave-d7lw krb5kdc[3684](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:47:47 dist-test-slave-d7lw krb5kdc[3684](info): set up 2 sockets
Sep 25 14:47:47 dist-test-slave-d7lw krb5kdc[3684](info): commencing operation
krb5kdc: starting...
W20260925 14:47:48.510746 22327 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:47:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:47:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:47:48.624413 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:40439
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:40439
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:48.702291 3697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:48.702473 3697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:48.702494 3697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:48.703557 3697 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:48.703589 3697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:48.703603 3697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:48.703615 3697 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:48.703626 3697 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:48.704963 3697 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:40439
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:40439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:48.705147 3697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:48.705296 3697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:48.709791 3702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:48.709802 3705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:48.709802 3703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:48.709960 3697 server_base.cc:1034] running on GCE node
I20260925 14:47:48.710304 3697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:48.710569 3697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:48.711750 3697 hybrid_clock.cc:648] HybridClock initialized: now 1790347668711721 us; error 26 us; skew 500 ppm
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3684](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347668, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:48.714107 3697 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:48.714530 3697 webserver.cc:460] Webserver started at http://127.21.205.254:35653/ using document root <none> and password file <none>
I20260925 14:47:48.714748 3697 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:48.714811 3697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:48.714947 3697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:48.716034 3697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "919a9be001774b27bdc85471df7160e7"
format_stamp: "Formatted at 2026-09-25 14:47:48 on dist-test-slave-d7lw"
server_key: "1808fe2b7c5980de2e2917856b8b2393"
server_key_iv: "1481a83df455d0b764189c414724e463"
server_key_version: "encryptionkey@0"
I20260925 14:47:48.716385 3697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "919a9be001774b27bdc85471df7160e7"
format_stamp: "Formatted at 2026-09-25 14:47:48 on dist-test-slave-d7lw"
server_key: "1808fe2b7c5980de2e2917856b8b2393"
server_key_iv: "1481a83df455d0b764189c414724e463"
server_key_version: "encryptionkey@0"
I20260925 14:47:48.717628 3697 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:47:48.718326 3712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:48.718499 3697 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:48.718578 3697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "919a9be001774b27bdc85471df7160e7"
format_stamp: "Formatted at 2026-09-25 14:47:48 on dist-test-slave-d7lw"
server_key: "1808fe2b7c5980de2e2917856b8b2393"
server_key_iv: "1481a83df455d0b764189c414724e463"
server_key_version: "encryptionkey@0"
I20260925 14:47:48.718628 3697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:48.730319 3697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:48.732034 3697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:48.732131 3697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:48.736096 3697 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:40439
I20260925 14:47:48.736167 3766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:40439 every 8 connection(s)
I20260925 14:47:48.736505 3697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:48.737305 3768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:48.738667 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3697
I20260925 14:47:48.738767 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:47:48.738981 22327 external_mini_cluster.cc:1428] Setting key 3222d4015673aaf404033daf41a109b9
I20260925 14:47:48.740100 3768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: Bootstrap starting.
I20260925 14:47:48.740734 3768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:48.741074 3768 log.cc:824] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:48.741789 3768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: No bootstrap required, opened a new log
I20260925 14:47:48.743973 3768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } }
I20260925 14:47:48.744153 3768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:48.744222 3768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 919a9be001774b27bdc85471df7160e7, State: Initialized, Role: FOLLOWER
I20260925 14:47:48.744359 3768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } }
I20260925 14:47:48.744469 3768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:48.744517 3768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:48.744593 3768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:48.745570 3768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } }
I20260925 14:47:48.745702 3768 leader_election.cc:302] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 919a9be001774b27bdc85471df7160e7; no voters:
I20260925 14:47:48.745891 3768 leader_election.cc:288] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [CANDIDATE]: Term 1 election: Requested vote from peers
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3684](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347668, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:48.746043 3774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:48.746594 3774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [term 1 LEADER]: Becoming Leader. State: Replica: 919a9be001774b27bdc85471df7160e7, State: Running, Role: LEADER
I20260925 14:47:48.747313 3774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } }
I20260925 14:47:48.746326 3768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:48.748128 3773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "919a9be001774b27bdc85471df7160e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } } }
I20260925 14:47:48.748452 3773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:48.748365 3775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 919a9be001774b27bdc85471df7160e7. Latest consensus state: current_term: 1 leader_uuid: "919a9be001774b27bdc85471df7160e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "919a9be001774b27bdc85471df7160e7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 40439 } } }
W20260925 14:47:48.749823 3788 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260925 14:47:48.748683 3775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7 [sys.catalog]: This master's current role is: LEADER
W20260925 14:47:48.749874 3788 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:47:48.750085 3790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:48.750244 3790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:48.751441 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:48.739449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45270 (local address 127.21.205.254:40439)
0925 14:47:48.739638 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:48.739649 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:48.739683 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:47:48.740219 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:48.740227 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:48.740283 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:48.740443 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:48.741082 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:48.742404 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:48.743130 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:48.743832 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:48.743939 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:48.746852 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:48.746856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:48.746856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:48.746881 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:48.749713 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:48.750107 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:48.750110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:48.750110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:48.750255 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:48.750484 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:48.750485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:48.750485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:48.750807 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:48.750948 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:48.751114 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 14:47:48.751248 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260925 14:47:48.752419 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:47:48.755081 3790 catalog_manager.cc:1347] Generated new cluster ID: 9be59f5d1d8f4f5894abc5eb05661cc6
I20260925 14:47:48.755146 3790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:47:48.788173 3790 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:47:48.788698 3790 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:48.790990 3790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 919a9be001774b27bdc85471df7160e7: Generated new TSK 0
I20260925 14:47:48.791167 3790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:47:48.806084 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3697
2026-09-25T14:47:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1305 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3795](info): set up 2 sockets
Sep 25 14:47:48 dist-test-slave-d7lw krb5kdc[3795](info): commencing operation
krb5kdc: starting...
W20260925 14:47:49.830624 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.006s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:47:49 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:47:49.939232 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:34029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:50.016398 3808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:50.016611 3808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:50.016633 3808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:50.017686 3808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:50.017719 3808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:50.017745 3808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:50.017757 3808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:50.017769 3808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:50.019075 3808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:34029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:50.019251 3808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:50.019413 3808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:50.024052 3814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:50.024230 3808 server_base.cc:1034] running on GCE node
W20260925 14:47:50.024075 3816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.024060 3813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:50.024513 3808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:50.024791 3808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:50.025951 3808 hybrid_clock.cc:648] HybridClock initialized: now 1790347670025947 us; error 35 us; skew 500 ppm
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:50.028342 3808 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:50.028769 3808 webserver.cc:460] Webserver started at http://127.21.205.254:43125/ using document root <none> and password file <none>
I20260925 14:47:50.028985 3808 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:50.029049 3808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:50.029189 3808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:50.030220 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "1b7b8084f80f426a9dccae26976370be"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f87497ebcefaba3b9de5e41c836bb974"
server_key_iv: "986193a8d7b035ce6cafee6273bcbf2c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.030637 3808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "1b7b8084f80f426a9dccae26976370be"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f87497ebcefaba3b9de5e41c836bb974"
server_key_iv: "986193a8d7b035ce6cafee6273bcbf2c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.031915 3808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:47:50.032624 3823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.032791 3808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:50.032889 3808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "1b7b8084f80f426a9dccae26976370be"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f87497ebcefaba3b9de5e41c836bb974"
server_key_iv: "986193a8d7b035ce6cafee6273bcbf2c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.032971 3808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:50.060331 3808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:50.061987 3808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:50.062141 3808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:50.065948 3808 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:34029
I20260925 14:47:50.065994 3876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:34029 every 8 connection(s)
I20260925 14:47:50.066327 3808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:50.067286 3877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:50.069468 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap starting.
I20260925 14:47:50.069949 3877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:50.070164 3877 log.cc:824] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:50.070751 3877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: No bootstrap required, opened a new log
I20260925 14:47:50.072283 3877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:50.072418 3877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:50.072455 3877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Initialized, Role: FOLLOWER
I20260925 14:47:50.072609 3877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:50.072686 3877 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:50.072708 3877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:50.072736 3877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:50.073437 3877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:50.073515 3877 leader_election.cc:302] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b7b8084f80f426a9dccae26976370be; no voters:
I20260925 14:47:50.073624 3877 leader_election.cc:288] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:47:50.073724 3880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:50.073820 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3808
I20260925 14:47:50.073873 3877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:50.073920 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:47:50.073935 3880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 LEADER]: Becoming Leader. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Running, Role: LEADER
I20260925 14:47:50.074028 3880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:50.074162 22327 external_mini_cluster.cc:1428] Setting key d25ebdc1e4d09011b7cfce36a941935e
I20260925 14:47:50.074683 3880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7b8084f80f426a9dccae26976370be. Latest consensus state: current_term: 1 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:50.074764 3880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:50.074914 3881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:50.074973 3893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:50.075001 3881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:50.075711 3893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:50.077219 3893 catalog_manager.cc:1347] Generated new cluster ID: 425217ae0ba24143bd6a9556cea62b07
I20260925 14:47:50.077272 3893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:50.086306 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.074583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.21.205.254:34029)
0925 14:47:50.075339 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.075348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.075376 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:47:50.075487 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.075494 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.075547 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.075679 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:50.076432 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.077751 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.078833 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.079442 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.079531 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.082153 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:50.082157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:50.082157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:50.082170 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:50.084738 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.085103 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.085106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.085107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.085239 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.085443 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.085444 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.085445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.085748 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:50.085902 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.085972 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.086098 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":94,"threads_started":1}
I20260925 14:47:50.098767 3893 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:47:50.099465 3893 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:50.104184 3893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Generated new TSK 0
I20260925 14:47:50.104375 3893 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:47:50.130465 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:50.208505 3904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:50.208684 3904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:50.208705 3904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:50.209774 3904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:50.209812 3904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:50.209838 3904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:47:50.211266 3904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:50.211442 3904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:50.211669 3904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:50.216572 3909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.216562 3912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.216562 3910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:50.216802 3904 server_base.cc:1034] running on GCE node
I20260925 14:47:50.217025 3904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:50.217257 3904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:50.218396 3904 hybrid_clock.cc:648] HybridClock initialized: now 1790347670218377 us; error 36 us; skew 500 ppm
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:50.220948 3904 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:47:50.221375 3904 webserver.cc:460] Webserver started at http://127.21.205.193:46261/ using document root <none> and password file <none>
I20260925 14:47:50.221565 3904 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:50.221604 3904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:50.221689 3904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:50.222630 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "6346a24370614fa3a9fd748a834e41f3"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f2d19cc04a7e3159969a97b797801d04"
server_key_iv: "15f7e12197cdb883f719fcbe0ddb861c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.222930 3904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "6346a24370614fa3a9fd748a834e41f3"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f2d19cc04a7e3159969a97b797801d04"
server_key_iv: "15f7e12197cdb883f719fcbe0ddb861c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.224179 3904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:47:50.224821 3920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.224994 3904 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:50.225080 3904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "6346a24370614fa3a9fd748a834e41f3"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f2d19cc04a7e3159969a97b797801d04"
server_key_iv: "15f7e12197cdb883f719fcbe0ddb861c"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.225152 3904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:50.249809 3904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:50.251579 3904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:50.251731 3904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:50.251968 3904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:50.252257 3904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:50.252292 3904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.252334 3904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:50.252362 3904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.258127 3904 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:36429
I20260925 14:47:50.258172 4033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:36429 every 8 connection(s)
I20260925 14:47:50.258460 3904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:50.265157 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 3904
I20260925 14:47:50.265264 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:47:50.265492 22327 external_mini_cluster.cc:1428] Setting key d8fbb6ea60541b73bcb0bd9dbdaa372e
I20260925 14:47:50.267204 4036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.259028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:46049)
0925 14:47:50.259175 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.259214 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.259700 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.259954 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.259957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.260505 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:50.260697 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.260701 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.261344 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.261345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.262056 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.262058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.262408 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.262409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.262508 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.262914 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:50.262921 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:50.264249 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:50.265846 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.265854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.265864 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.266124 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.266333 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.266333 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.266333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.266452 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.266751 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:50.266755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:50.266865 (+ 110us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.266945 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.267054 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":81,"threads_started":1}
I20260925 14:47:50.267218 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.259071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:46049 (local address 127.21.205.254:34029)
0925 14:47:50.259115 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.259117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.259298 (+ 181us) server_negotiation.cc:407] Connection header received
0925 14:47:50.259811 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.259813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.259823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.259898 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:50.260774 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.261258 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.262125 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.262328 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.262376 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.264359 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:50.264361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:50.264361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:50.264368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:50.265766 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.266215 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.266216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.266217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.266274 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.266517 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.266518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.266518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.266684 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:50.266757 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.266991 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.267082 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:47:50.268065 4034 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:50.268164 4034 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:50.268335 4034 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:50.268867 3841 ts_manager.cc:194] Registered new tserver with Master: 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429)
I20260925 14:47:50.269707 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:46049
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:47:50.288553 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:50.367230 4039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:50.367467 4039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:50.367504 4039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:50.368659 4039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:50.368736 4039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:50.368781 4039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:47:50.370203 4039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:50.370474 4039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:50.370697 4039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:50.375651 4045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.375684 4044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.375684 4047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:50.375927 4039 server_base.cc:1034] running on GCE node
I20260925 14:47:50.376224 4039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:50.376469 4039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:50.377636 4039 hybrid_clock.cc:648] HybridClock initialized: now 1790347670377632 us; error 49 us; skew 500 ppm
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:50.379999 4039 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:47:50.380496 4039 webserver.cc:460] Webserver started at http://127.21.205.194:42835/ using document root <none> and password file <none>
I20260925 14:47:50.380731 4039 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:50.380797 4039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:50.380930 4039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:50.381932 4039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "5c97749581b44783ac1127112e6104ea"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "e2156945ad73ffec25a2896c42dacec0"
server_key_iv: "7a9606cad94ed89713fdab6b9c8adfe4"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.382288 4039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "5c97749581b44783ac1127112e6104ea"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "e2156945ad73ffec25a2896c42dacec0"
server_key_iv: "7a9606cad94ed89713fdab6b9c8adfe4"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.383605 4039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:47:50.384379 4054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.384560 4039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.384654 4039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "5c97749581b44783ac1127112e6104ea"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "e2156945ad73ffec25a2896c42dacec0"
server_key_iv: "7a9606cad94ed89713fdab6b9c8adfe4"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.384742 4039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:50.401979 4039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:50.403944 4039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:50.404110 4039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:50.404318 4039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:50.404620 4039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:50.404680 4039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.404719 4039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:50.404744 4039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.410360 4039 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:39955
I20260925 14:47:50.410409 4169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:39955 every 8 connection(s)
I20260925 14:47:50.410696 4039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:47:50.413373 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4039
I20260925 14:47:50.413461 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:47:50.413666 22327 external_mini_cluster.cc:1428] Setting key c83f436f8759d5c60f88a34668f0e4ea
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:50.420323 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.411412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:52435 (local address 127.21.205.254:34029)
0925 14:47:50.411511 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.411513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.411663 (+ 150us) server_negotiation.cc:407] Connection header received
0925 14:47:50.412211 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.412213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.412230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.412280 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:50.413192 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.413844 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.414892 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.415101 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.415185 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.417565 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:50.417566 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:50.417566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:50.417573 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:50.418893 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.419375 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.419375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.419376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.419424 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.419698 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.419699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.419699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.419836 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:50.419902 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.420207 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.420266 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 14:47:50.420599 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.411364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:52435)
0925 14:47:50.411541 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.411579 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.412101 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.412343 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.412347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.412907 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:50.413084 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.413088 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.413937 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.413938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.414797 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.414800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.415195 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.415196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.415324 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.415859 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:50.415868 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:50.417477 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:50.418974 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.418980 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.418983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.419301 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.419489 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.419490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.419491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.419630 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.419903 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:50.419911 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:50.420079 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.420196 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.420349 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260925 14:47:50.421089 4170 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:50.421172 4170 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:50.421336 4170 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:50.421729 3841 ts_manager.cc:194] Registered new tserver with Master: 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
I20260925 14:47:50.422044 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:52435
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:47:50.436882 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:50.516130 4176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:50.516350 4176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:50.516397 4176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:50.517544 4176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:50.517614 4176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:50.517659 4176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:47:50.519080 4176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:50.519337 4176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:50.519570 4176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:50.524432 4184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.524457 4182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:50.524457 4181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:50.524765 4176 server_base.cc:1034] running on GCE node
I20260925 14:47:50.524981 4176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:50.525221 4176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:50.526362 4176 hybrid_clock.cc:648] HybridClock initialized: now 1790347670526348 us; error 28 us; skew 500 ppm
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:50.528798 4176 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:47:50.529209 4176 webserver.cc:460] Webserver started at http://127.21.205.195:36717/ using document root <none> and password file <none>
I20260925 14:47:50.529446 4176 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:50.529515 4176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:50.529663 4176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:47:50.530668 4176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "2271f634696a4fc0a5eb62c403df3367"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "2be6e522d8d6d33efd1a86f62e00d3ba"
server_key_iv: "3379d2ecdc9d2d6ce85811f7a8369f29"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.531039 4176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "2271f634696a4fc0a5eb62c403df3367"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "2be6e522d8d6d33efd1a86f62e00d3ba"
server_key_iv: "3379d2ecdc9d2d6ce85811f7a8369f29"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.532416 4176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260925 14:47:50.533133 4191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.533294 4176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:50.533390 4176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "2271f634696a4fc0a5eb62c403df3367"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "2be6e522d8d6d33efd1a86f62e00d3ba"
server_key_iv: "3379d2ecdc9d2d6ce85811f7a8369f29"
server_key_version: "encryptionkey@0"
I20260925 14:47:50.533471 4176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:50.544430 4176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:50.546128 4176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:50.546280 4176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:50.546490 4176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:50.546769 4176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:50.546823 4176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.546870 4176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:50.546898 4176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:50.552698 4176 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:40481
I20260925 14:47:50.552747 4306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:40481 every 8 connection(s)
I20260925 14:47:50.553035 4176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:47:50 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347670, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:50.561285 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.553702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:41155 (local address 127.21.205.254:34029)
0925 14:47:50.553779 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.553782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.553927 (+ 145us) server_negotiation.cc:407] Connection header received
0925 14:47:50.554365 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.554366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.554375 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.554427 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:50.555267 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.555773 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.556804 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.556990 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.557039 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.558932 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:50.558933 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:50.558934 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:50.558940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:50.560117 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.560457 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.560458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.560458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.560503 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.560709 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.560709 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.560709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.560803 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:50.560849 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.561067 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.561184 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 14:47:50.561293 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.553647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:41155)
0925 14:47:50.553792 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.553825 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.554283 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.554476 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.554479 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.555006 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:50.555190 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.555194 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.555889 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.555890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.556725 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.556727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.557061 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.557062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.557158 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.557567 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:50.557574 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:50.558844 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:50.560181 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.560187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.560188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.560393 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.560568 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:50.560568 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:50.560569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:50.560657 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:50.560855 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:50.560859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:50.560962 (+ 103us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.561039 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.561146 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":89,"threads_started":1}
I20260925 14:47:50.561276 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4176
I20260925 14:47:50.561421 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:47:50.561585 22327 external_mini_cluster.cc:1428] Setting key 01cccf08f2fcf914d730acdc042af990
I20260925 14:47:50.562134 4307 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:50.562239 4307 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:50.562474 4307 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:50.562879 3841 ts_manager.cc:194] Registered new tserver with Master: 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481)
I20260925 14:47:50.563259 3841 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:41155
I20260925 14:47:50.573450 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:47:50.580713 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.574808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40598 (local address 127.21.205.254:34029)
0925 14:47:50.574877 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.574879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.574990 (+ 111us) server_negotiation.cc:407] Connection header received
0925 14:47:50.575051 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.575052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.575061 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.575105 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:50.575798 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.576275 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.577019 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.577224 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.577298 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.578374 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:50.578375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:50.578376 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:50.578382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:50.579504 (+ 1122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.579875 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.579876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.579876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.579943 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:50.580147 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:50.580148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:50.580148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:50.580288 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:50.580388 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.580540 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.580616 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:47:50.582185 3841 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:47:50.583760 3841 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:47:50.590301 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.586244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40928 (local address 127.21.205.195:40481)
0925 14:47:50.586407 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.586412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.586422 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:47:50.586464 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.586466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.586482 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.586811 (+ 329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:50.586870 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.587993 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589218 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.589858 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589958 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590014 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.590174 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.590215 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260925 14:47:50.590296 4315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.585875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40481 (local address 127.0.0.1:40928)
0925 14:47:50.586290 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.586298 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.586332 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.586612 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.586613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.586616 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:50.586669 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.586671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.588095 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.588096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.589150 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.589152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.589960 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.590045 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590065 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.590154 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.590212 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":13,"thread_start_us":44,"threads_started":1}
I20260925 14:47:50.590467 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.586138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53230 (local address 127.21.205.193:36429)
0925 14:47:50.586415 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.586418 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.586425 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:47:50.586456 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.586457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.586469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.586554 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:50.586717 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.587587 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589171 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.589881 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589953 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590007 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.590335 (+ 328us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.590391 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":80,"threads_started":1}
I20260925 14:47:50.590458 4317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.585875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39955 (local address 127.0.0.1:33988)
0925 14:47:50.586029 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.586090 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.586155 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.586929 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.586930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.586932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:50.586975 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.586976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.588325 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.588326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.589322 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.589325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.590251 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.590252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.590308 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590312 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.590361 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.590392 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":61,"threads_started":1}
I20260925 14:47:50.590298 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.585886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:36429 (local address 127.0.0.1:53230)
0925 14:47:50.586107 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.586119 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.586169 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.586569 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.586589 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.586591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:50.586656 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.586658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.587694 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.587695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.589080 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.589083 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.590042 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.590043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.590117 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590121 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.590168 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.590214 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":51,"threads_started":1}
I20260925 14:47:50.590723 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.586054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.21.205.194:39955)
0925 14:47:50.586769 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.586773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.586782 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:47:50.586821 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.586823 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.586837 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.586917 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:50.587039 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.588110 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.589419 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.590108 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.590174 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.590218 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.590566 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.590613 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":84,"threads_started":1}
I20260925 14:47:50.591321 4103 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e (DEFAULT_TABLE table=test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:50.591320 3968 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e (DEFAULT_TABLE table=test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:50.591646 4103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cab4a7f4ebfb5630b04be130a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:50.591652 3968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cab4a7f4ebfb5630b04be130a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:50.594190 4321 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Bootstrap starting.
I20260925 14:47:50.594216 4322 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Bootstrap starting.
I20260925 14:47:50.594828 4322 tablet_bootstrap.cc:654] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:50.594866 4321 tablet_bootstrap.cc:654] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:50.594947 4240 tablet_service.cc:1467] Processing CreateTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e (DEFAULT_TABLE table=test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:50.595291 4240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40864cab4a7f4ebfb5630b04be130a1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:50.595443 4321 log.cc:824] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:50.595441 4322 log.cc:824] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:50.596282 4321 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: No bootstrap required, opened a new log
I20260925 14:47:50.596320 4322 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: No bootstrap required, opened a new log
I20260925 14:47:50.596355 4321 ts_tablet_manager.cc:1397] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:50.596411 4322 ts_tablet_manager.cc:1397] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:50.597910 4322 raft_consensus.cc:348] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.598054 4322 raft_consensus.cc:374] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:50.598098 4322 raft_consensus.cc:729] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6346a24370614fa3a9fd748a834e41f3, State: Initialized, Role: FOLLOWER
I20260925 14:47:50.598237 4322 consensus_queue.cc:260] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.598517 4322 ts_tablet_manager.cc:1428] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:50.598526 4034 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
I20260925 14:47:50.598359 4321 raft_consensus.cc:348] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.598841 4321 raft_consensus.cc:374] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:50.598906 4321 raft_consensus.cc:729] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c97749581b44783ac1127112e6104ea, State: Initialized, Role: FOLLOWER
I20260925 14:47:50.599007 4321 consensus_queue.cc:260] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.599272 4321 ts_tablet_manager.cc:1428] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:47:50.599282 4170 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
I20260925 14:47:50.599586 4327 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Bootstrap starting.
I20260925 14:47:50.600190 4327 tablet_bootstrap.cc:654] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:50.600666 4327 log.cc:824] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:50.601223 4327 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: No bootstrap required, opened a new log
I20260925 14:47:50.601274 4327 ts_tablet_manager.cc:1397] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:50.602432 4327 raft_consensus.cc:348] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.602541 4327 raft_consensus.cc:374] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:50.602560 4327 raft_consensus.cc:729] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2271f634696a4fc0a5eb62c403df3367, State: Initialized, Role: FOLLOWER
I20260925 14:47:50.602624 4327 consensus_queue.cc:260] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.602794 4327 ts_tablet_manager.cc:1428] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:50.602905 4307 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
W20260925 14:47:50.661352 4171 tablet.cc:2369] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:47:50.759205 4035 tablet.cc:2369] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:47:50.803699 4308 tablet.cc:2369] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:47:50.935541 4326 raft_consensus.cc:482] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:50.935694 4326 raft_consensus.cc:504] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.936115 4326 leader_election.cc:288] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
I20260925 14:47:50.939072 4318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.936377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:39639 (local address 127.21.205.194:39955)
0925 14:47:50.936427 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.936429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.936525 (+ 96us) server_negotiation.cc:407] Connection header received
0925 14:47:50.936641 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.936643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.936655 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.936736 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:50.936900 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.937369 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938350 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.938706 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938794 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.938803 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.938957 (+ 154us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.938995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:47:50.939077 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.936396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:45141 (local address 127.21.205.195:40481)
0925 14:47:50.936441 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:50.936443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:50.936507 (+ 64us) server_negotiation.cc:407] Connection header received
0925 14:47:50.936641 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:50.936642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:50.936653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:50.936731 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:50.936913 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.937371 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938346 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.938727 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938812 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.938820 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:50.938968 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 14:47:50.939005 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:47:50.939113 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.936294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40481 (local address 127.21.205.193:45141)
0925 14:47:50.936448 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.936457 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.936552 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.936736 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.936737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.936739 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:50.936831 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.936833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.937453 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.937454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.938254 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.938256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.938813 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.938895 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.938898 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.938963 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.939007 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":14,"thread_start_us":69,"threads_started":1}
I20260925 14:47:50.939119 4332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:50.936302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39955 (local address 127.21.205.193:39639)
0925 14:47:50.936456 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:50.936466 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:50.936552 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:50.936741 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:50.936742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:50.936743 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:50.936812 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.936814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.937455 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.937456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.938255 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:50.938256 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:50.938794 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:50.938795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:50.938867 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:50.938871 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:50.938949 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 14:47:50.938995 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":52,"threads_started":1}
I20260925 14:47:50.939500 4123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cab4a7f4ebfb5630b04be130a1e" candidate_uuid: "6346a24370614fa3a9fd748a834e41f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c97749581b44783ac1127112e6104ea" is_pre_election: true
I20260925 14:47:50.939502 4261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cab4a7f4ebfb5630b04be130a1e" candidate_uuid: "6346a24370614fa3a9fd748a834e41f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2271f634696a4fc0a5eb62c403df3367" is_pre_election: true
I20260925 14:47:50.939705 4261 raft_consensus.cc:2454] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6346a24370614fa3a9fd748a834e41f3 in term 0.
I20260925 14:47:50.939705 4123 raft_consensus.cc:2454] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6346a24370614fa3a9fd748a834e41f3 in term 0.
I20260925 14:47:50.939982 3921 leader_election.cc:302] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c97749581b44783ac1127112e6104ea, 6346a24370614fa3a9fd748a834e41f3; no voters:
I20260925 14:47:50.940114 4326 raft_consensus.cc:2781] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:47:50.940199 4326 raft_consensus.cc:482] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:47:50.940233 4326 raft_consensus.cc:3037] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:50.940904 4326 raft_consensus.cc:504] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.941031 4326 leader_election.cc:288] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 1 election: Requested vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
I20260925 14:47:50.941216 4123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cab4a7f4ebfb5630b04be130a1e" candidate_uuid: "6346a24370614fa3a9fd748a834e41f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c97749581b44783ac1127112e6104ea"
I20260925 14:47:50.941219 4261 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40864cab4a7f4ebfb5630b04be130a1e" candidate_uuid: "6346a24370614fa3a9fd748a834e41f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2271f634696a4fc0a5eb62c403df3367"
I20260925 14:47:50.941305 4123 raft_consensus.cc:3037] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:50.941300 4261 raft_consensus.cc:3037] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:50.942019 4261 raft_consensus.cc:2454] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6346a24370614fa3a9fd748a834e41f3 in term 1.
I20260925 14:47:50.942101 4123 raft_consensus.cc:2454] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6346a24370614fa3a9fd748a834e41f3 in term 1.
I20260925 14:47:50.942204 3923 leader_election.cc:302] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2271f634696a4fc0a5eb62c403df3367, 6346a24370614fa3a9fd748a834e41f3; no voters:
I20260925 14:47:50.942308 4326 raft_consensus.cc:2781] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:50.942461 4326 raft_consensus.cc:686] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 LEADER]: Becoming Leader. State: Replica: 6346a24370614fa3a9fd748a834e41f3, State: Running, Role: LEADER
I20260925 14:47:50.942576 4326 consensus_queue.cc:237] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:50.943378 3841 catalog_manager.cc:5495] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "6346a24370614fa3a9fd748a834e41f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } health_report { overall_health: HEALTHY } } }
I20260925 14:47:51.031495 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.029344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53234 (local address 127.21.205.193:36429)
0925 14:47:51.029411 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.029413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.029421 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:47:51.029491 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.029491 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.029511 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.029566 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:47:51.029711 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.030141 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.030843 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.031016 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.031062 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.031190 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:47:51.031285 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:47:51.031342 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.031375 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.031390 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:47:51.033243 4261 raft_consensus.cc:1261] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Refusing update from remote peer 6346a24370614fa3a9fd748a834e41f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:51.033243 4123 raft_consensus.cc:1261] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Refusing update from remote peer 6346a24370614fa3a9fd748a834e41f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:51.033604 4333 consensus_queue.cc:1035] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:51.033771 4326 consensus_queue.cc:1035] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:51.036083 4336 mvcc.cc:205] Tried to move back new op lower bound from 7333264060549464064 to 7333264060181426176. Current Snapshot: MvccSnapshot[applied={T|T < 7333264060549464064}]
I20260925 14:47:51.036238 4337 mvcc.cc:205] Tried to move back new op lower bound from 7333264060549464064 to 7333264060181426176. Current Snapshot: MvccSnapshot[applied={T|T < 7333264060549464064}]
I20260925 14:47:51.036266 4338 mvcc.cc:205] Tried to move back new op lower bound from 7333264060549464064 to 7333264060181426176. Current Snapshot: MvccSnapshot[applied={T|T < 7333264060549464064}]
I20260925 14:47:51.038749 3839 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:51.038837 3839 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:51.039553 3839 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Sending DeleteTablet for 3 replicas of tablet 40864cab4a7f4ebfb5630b04be130a1e
I20260925 14:47:51.039891 4240 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:40928
I20260925 14:47:51.039888 3968 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:53230
I20260925 14:47:51.040125 4346 tablet_replica.cc:330] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: stopping tablet replica
I20260925 14:47:51.040140 4103 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:33988
I20260925 14:47:51.040230 4346 raft_consensus.cc:2229] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:47:51.040339 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3904
I20260925 14:47:51.040359 4346 raft_consensus.cc:2258] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.040373 4349 tablet_replica.cc:330] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: stopping tablet replica
I20260925 14:47:51.040483 4349 raft_consensus.cc:2229] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:51.040584 4349 raft_consensus.cc:2258] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.040910 4347 tablet_replica.cc:330] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: stopping tablet replica
I20260925 14:47:51.041010 4347 raft_consensus.cc:2229] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:51.041116 4347 raft_consensus.cc:2258] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.041543 4347 ts_tablet_manager.cc:1905] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:51.042086 4349 ts_tablet_manager.cc:1905] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:51.043349 4347 ts_tablet_manager.cc:1918] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:51.043431 4347 log.cc:1196] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/40864cab4a7f4ebfb5630b04be130a1e
I20260925 14:47:51.043752 4347 ts_tablet_manager.cc:1939] T 40864cab4a7f4ebfb5630b04be130a1e P 2271f634696a4fc0a5eb62c403df3367: Deleting consensus metadata
I20260925 14:47:51.043867 4349 ts_tablet_manager.cc:1918] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:51.043948 4349 log.cc:1196] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/40864cab4a7f4ebfb5630b04be130a1e
I20260925 14:47:51.044173 3826 catalog_manager.cc:4841] TS 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): tablet 40864cab4a7f4ebfb5630b04be130a1e (table test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]) successfully deleted
I20260925 14:47:51.044226 4349 ts_tablet_manager.cc:1939] T 40864cab4a7f4ebfb5630b04be130a1e P 5c97749581b44783ac1127112e6104ea: Deleting consensus metadata
I20260925 14:47:51.044696 3824 catalog_manager.cc:4841] TS 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): tablet 40864cab4a7f4ebfb5630b04be130a1e (table test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]) successfully deleted
W20260925 14:47:51.047112 3827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:36429 (error 108)
I20260925 14:47:51.047367 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4039
I20260925 14:47:51.047542 4316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.047415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:36429 (local address 127.0.0.1:53250)
0925 14:47:51.047473 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.047507 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 14:47:51.048235 3827 catalog_manager.cc:4584] TS 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40864cab4a7f4ebfb5630b04be130a1e: Network error: Client connection negotiation failed: client connection to 127.21.205.193:36429: connect: Connection refused (error 111)
I20260925 14:47:51.053448 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4176
I20260925 14:47:51.058175 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 3808
I20260925 14:47:51.063202 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--webserver_interface=127.21.205.254
--webserver_port=43125
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:34029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:51.140533 4351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.140753 4351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.140791 4351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.141906 4351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:51.141983 4351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.142006 4351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.142037 4351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:51.142060 4351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:51.143440 4351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:34029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.143685 4351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.143931 4351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:51.156957 4358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.156919 4357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.157207 4351 server_base.cc:1034] running on GCE node
W20260925 14:47:51.156919 4360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.157764 4351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.158073 4351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.159243 4351 hybrid_clock.cc:648] HybridClock initialized: now 1790347671159231 us; error 40 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.165362 4351 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:51.165763 4351 webserver.cc:460] Webserver started at http://127.21.205.254:43125/ using document root <none> and password file <none>
I20260925 14:47:51.165990 4351 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.166067 4351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.167430 4351 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:51.168123 4367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.169289 4351 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:51.169391 4351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "1b7b8084f80f426a9dccae26976370be"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f87497ebcefaba3b9de5e41c836bb974"
server_key_iv: "986193a8d7b035ce6cafee6273bcbf2c"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.169744 4351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:51.180292 4351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:51.181967 4351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:51.182072 4351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:51.185844 4351 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:34029
I20260925 14:47:51.185926 4419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:34029 every 8 connection(s)
I20260925 14:47:51.186297 4351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:51.187712 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4351
I20260925 14:47:51.187976 4420 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:47:51.188089 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:36429
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=46261
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:51.189065 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap starting.
I20260925 14:47:51.197693 4420 log.cc:824] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:51.199518 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:47:51.199846 4420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap complete.
I20260925 14:47:51.202231 4420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.202452 4420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.202617 4420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.202674 4420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:51.202741 4420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:51.202806 4420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 1 FOLLOWER]: Advancing to term 2
I20260925 14:47:51.203842 4420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.203969 4420 leader_election.cc:302] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b7b8084f80f426a9dccae26976370be; no voters:
I20260925 14:47:51.204116 4420 leader_election.cc:288] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 14:47:51.204149 4424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Leader election won for term 2
I20260925 14:47:51.204255 4424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 LEADER]: Becoming Leader. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Running, Role: LEADER
I20260925 14:47:51.204372 4420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:51.204392 4424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.204874 4426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7b8084f80f426a9dccae26976370be. Latest consensus state: current_term: 2 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:51.204885 4425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:51.205015 4425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:51.204969 4426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:51.205265 4432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:51.206239 4432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:51.206383 4432 catalog_manager.cc:1259] Loaded cluster ID: 425217ae0ba24143bd6a9556cea62b07
I20260925 14:47:51.206650 4432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:47:51.206920 4432 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 14:47:51.211488 4432 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 14:47:51.213945 4432 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:47:51.214136 4432 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Loaded TSK: 0
I20260925 14:47:51.214340 4432 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:47:51.269078 4422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.269282 4422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.269320 4422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.270419 4422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.270481 4422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.270527 4422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:47:51.272106 4422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:36429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=46261
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.272344 4422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.272612 4422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:51.278450 4448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.278456 4450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.278460 4447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.279063 4422 server_base.cc:1034] running on GCE node
I20260925 14:47:51.279266 4422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.279538 4422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.280685 4422 hybrid_clock.cc:648] HybridClock initialized: now 1790347671280671 us; error 31 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.283043 4422 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:47:51.283505 4422 webserver.cc:460] Webserver started at http://127.21.205.193:46261/ using document root <none> and password file <none>
I20260925 14:47:51.283746 4422 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.283842 4422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.285157 4422 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:51.285868 4457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.286037 4422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:51.286135 4422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "6346a24370614fa3a9fd748a834e41f3"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f2d19cc04a7e3159969a97b797801d04"
server_key_iv: "15f7e12197cdb883f719fcbe0ddb861c"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.286489 4422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:51.301515 4422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:51.303544 4422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:51.303692 4422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:51.303952 4422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:51.304337 4465 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 14:47:51.306093 4422 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 14:47:51.306147 4422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 14:47:51.306190 4422 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 14:47:51.306775 4422 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 14:47:51.306830 4422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:51.306906 4465 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Bootstrap starting.
I20260925 14:47:51.313853 4422 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:36429
I20260925 14:47:51.313913 4574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:36429 every 8 connection(s)
I20260925 14:47:51.314266 4422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:47:51.318527 4465 log.cc:824] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:51.321676 4465 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:47:51.321975 4465 tablet_bootstrap.cc:492] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Bootstrap complete.
I20260925 14:47:51.323102 4465 ts_tablet_manager.cc:1397] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260925 14:47:51.323241 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4422
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:51.323539 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:39955
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=42835
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:51.324373 4465 raft_consensus.cc:348] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.324747 4465 raft_consensus.cc:729] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6346a24370614fa3a9fd748a834e41f3, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.324932 4465 consensus_queue.cc:260] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.325229 4465 ts_tablet_manager.cc:1428] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:51.328739 4577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.315122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:53913)
0925 14:47:51.315337 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.315382 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.315925 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.316345 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.316349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.316627 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:51.316808 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.316812 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.318007 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.318009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.318870 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.318872 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.319502 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.319503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.319633 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.321954 (+ 2321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:51.321964 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:51.324441 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:51.326860 (+ 2419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.326866 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.326868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.327151 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.327462 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.327463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.327464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.327595 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.328102 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:51.328109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:51.328262 (+ 153us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.328375 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.328543 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
I20260925 14:47:51.328897 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.315200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:53913 (local address 127.21.205.254:34029)
0925 14:47:51.315966 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.315975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.316012 (+ 37us) server_negotiation.cc:407] Connection header received
0925 14:47:51.316139 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.316143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.316184 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.316317 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:51.316930 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.317883 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.318975 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.319394 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.319512 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.324977 (+ 5465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:51.324981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:51.324982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:51.324994 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:51.326770 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.327252 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.327255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.327255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.327375 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.327670 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.327671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.327672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.328022 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:51.328115 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.328465 (+ 350us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.328624 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":716,"thread_start_us":117,"threads_started":1}
I20260925 14:47:51.329900 4575 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:51.330011 4575 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:51.330201 4575 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:51.330974 4384 ts_manager.cc:194] Registered new tserver with Master: 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429)
I20260925 14:47:51.332280 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:53913
I20260925 14:47:51.333326 4575 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
I20260925 14:47:51.340000 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.331623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:36429 (local address 127.0.0.1:53266)
0925 14:47:51.335902 (+ 4279us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.335939 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.336010 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.336199 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.336202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.336204 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:51.336261 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.336263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.337503 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.337504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.338802 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.338805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.339673 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.339674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.339751 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.339763 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.339876 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.339924 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4249,"thread_start_us":85,"threads_started":1}
I20260925 14:47:51.340008 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.331888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53266 (local address 127.21.205.193:36429)
0925 14:47:51.332022 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.332026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.336014 (+ 3988us) server_negotiation.cc:407] Connection header received
0925 14:47:51.336085 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.336087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.336103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.336186 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:51.336329 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.337416 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.338903 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.339586 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.339657 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.339704 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.339905 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.339944 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260925 14:47:51.340775 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet 40864cab4a7f4ebfb5630b04be130a1e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:53266
I20260925 14:47:51.341312 4584 tablet_replica.cc:330] stopping tablet replica
I20260925 14:47:51.341455 4584 raft_consensus.cc:2229] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:51.341526 4584 raft_consensus.cc:2258] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.341825 4584 ts_tablet_manager.cc:1905] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:51.343580 4584 ts_tablet_manager.cc:1918] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:51.343640 4584 log.cc:1196] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/40864cab4a7f4ebfb5630b04be130a1e
I20260925 14:47:51.343894 4584 ts_tablet_manager.cc:1939] T 40864cab4a7f4ebfb5630b04be130a1e P 6346a24370614fa3a9fd748a834e41f3: Deleting consensus metadata
I20260925 14:47:51.344378 4371 catalog_manager.cc:4841] TS 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429): tablet 40864cab4a7f4ebfb5630b04be130a1e (table test-table [id=c86b956d002d4c6fa8554e8fd2e8a16f]) successfully deleted
W20260925 14:47:51.408149 4580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.408352 4580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.408403 4580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.409476 4580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.409540 4580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.409583 4580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:47:51.410961 4580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:39955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=42835
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.411221 4580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.411441 4580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:51.416316 4589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.416333 4592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.416368 4590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.416536 4580 server_base.cc:1034] running on GCE node
I20260925 14:47:51.416769 4580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.417011 4580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.418145 4580 hybrid_clock.cc:648] HybridClock initialized: now 1790347671418125 us; error 37 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.420432 4580 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:47:51.420778 4580 webserver.cc:460] Webserver started at http://127.21.205.194:42835/ using document root <none> and password file <none>
I20260925 14:47:51.420955 4580 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.421003 4580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.422245 4580 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:51.422770 4599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.422958 4580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:51.423056 4580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "5c97749581b44783ac1127112e6104ea"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "e2156945ad73ffec25a2896c42dacec0"
server_key_iv: "7a9606cad94ed89713fdab6b9c8adfe4"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.423419 4580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:51.439522 4580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:51.441435 4580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:51.441606 4580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:51.441825 4580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:51.442119 4580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:51.442169 4580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.442227 4580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:51.442255 4580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.447955 4580 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:39955
I20260925 14:47:51.447989 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:39955 every 8 connection(s)
I20260925 14:47:51.448307 4580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:47:51.449911 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4580
I20260925 14:47:51.450256 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:40481
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=36717
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:51.458117 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.448943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:49313 (local address 127.21.205.254:34029)
0925 14:47:51.448981 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.448983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.449217 (+ 234us) server_negotiation.cc:407] Connection header received
0925 14:47:51.449665 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.449666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.449676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.449731 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:51.451042 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.451555 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.452627 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.452842 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.452935 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.454944 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:51.454946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:51.454946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:51.454953 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:51.456465 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.456970 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.456971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.456972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.457036 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.457417 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.457418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.457419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.457563 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:51.457638 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.457940 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.458030 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 14:47:51.458438 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.448926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:49313)
0925 14:47:51.449079 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.449116 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.449586 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.449843 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.449847 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.450667 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:51.450948 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.450955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.451657 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.451658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.452532 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.452535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.452952 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.452953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.453070 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.453562 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:51.453570 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:51.454845 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:51.456553 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.456560 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.456562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.456893 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.457162 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.457163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.457164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.457313 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.457661 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:51.457668 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:51.457786 (+ 118us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.457918 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.458065 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260925 14:47:51.458864 4716 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:51.458966 4716 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:51.459244 4716 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:51.459698 4384 ts_manager.cc:194] Registered new tserver with Master: 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
I20260925 14:47:51.460080 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:49313
W20260925 14:47:51.531229 4719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.531405 4719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.531426 4719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.532572 4719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.532610 4719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.532636 4719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:47:51.534021 4719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:40481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=36717
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.534269 4719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.534467 4719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:51.539161 4727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.539263 4719 server_base.cc:1034] running on GCE node
W20260925 14:47:51.539124 4725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.539142 4724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.539548 4719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.539829 4719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.540980 4719 hybrid_clock.cc:648] HybridClock initialized: now 1790347671540968 us; error 27 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.543298 4719 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:47:51.543748 4719 webserver.cc:460] Webserver started at http://127.21.205.195:36717/ using document root <none> and password file <none>
I20260925 14:47:51.543985 4719 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.544068 4719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.545404 4719 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:51.545944 4734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.546092 4719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:47:51.546190 4719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "2271f634696a4fc0a5eb62c403df3367"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "2be6e522d8d6d33efd1a86f62e00d3ba"
server_key_iv: "3379d2ecdc9d2d6ce85811f7a8369f29"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.546494 4719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:51.564553 4719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:51.566851 4719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:51.567005 4719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:51.567227 4719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:51.567551 4719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:51.567597 4719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.567637 4719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:51.567667 4719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.574131 4719 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:40481
I20260925 14:47:51.574211 4849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:40481 every 8 connection(s)
I20260925 14:47:51.574553 4719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:47:51.575109 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4719
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:47:51.582347 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.575621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.21.205.254:34029)
0925 14:47:51.575726 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.575727 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.575733 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:47:51.575759 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.575759 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.575773 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.575857 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:51.576366 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.577212 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.577965 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.578246 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.578312 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.579614 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:51.579615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:51.579615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:51.579621 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:51.581002 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.581343 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.581344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.581345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.581416 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.581738 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.581739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.581739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.581891 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:51.582028 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.582123 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.582236 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":77,"threads_started":1}
I20260925 14:47:51.584587 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.575578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:57675 (local address 127.21.205.254:34029)
0925 14:47:51.575652 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.575654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.575665 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:47:51.576204 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.576205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.576215 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.576267 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:51.577398 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.577847 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.578969 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.579173 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.579254 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.581658 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:51.581659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:51.581660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:51.581665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:51.583266 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.583655 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.583656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.583656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.583719 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:51.583992 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:51.583993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:51.583994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:51.584100 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:51.584172 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.584437 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.584510 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 14:47:51.584755 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.575267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:57675)
0925 14:47:51.575438 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.575488 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.576109 (+ 621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.576337 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.576341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.577063 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:51.577324 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.577331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.577949 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.577950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.578856 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.578858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.579266 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.579267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.579408 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.579924 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:51.579931 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:51.581564 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:51.583335 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.583341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.583343 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.583547 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.583810 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:51.583811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:51.583812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:51.583924 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:51.584166 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:51.584173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:51.584300 (+ 127us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.584397 (+ 97us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.584541 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260925 14:47:51.585227 4850 heartbeater.cc:347] Connected to a master server at 127.21.205.254:34029
I20260925 14:47:51.585326 4850 heartbeater.cc:464] Registering TS with master...
I20260925 14:47:51.585512 4850 heartbeater.cc:511] Master 127.21.205.254:34029 requested a full tablet report, sending...
I20260925 14:47:51.585865 4384 ts_manager.cc:194] Registered new tserver with Master: 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481)
I20260925 14:47:51.586210 4384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:57675
I20260925 14:47:51.594026 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:47:51.596527 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.594418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.21.205.254:34029)
0925 14:47:51.594466 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.594468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.594567 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:47:51.594611 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.594611 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.594639 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.594694 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:47:51.594793 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.595204 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.595911 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.596074 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.596122 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.596292 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:47:51.596374 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:47:51.596437 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.596473 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.596485 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 14:47:51.596782 4384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40626:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:47:51.598352 4384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:47:51.600446 4492 tablet_service.cc:1467] Processing CreateTablet for tablet acbe2a5e3a87402fae9ae791adf85c94 (DEFAULT_TABLE table=test-table [id=401f9c3818924eecbe20d5d3d9308fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:51.600646 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acbe2a5e3a87402fae9ae791adf85c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:51.601958 4465 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Bootstrap starting.
I20260925 14:47:51.602274 4465 tablet_bootstrap.cc:654] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:51.602864 4465 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: No bootstrap required, opened a new log
I20260925 14:47:51.602926 4465 ts_tablet_manager.cc:1397] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:51.603022 4465 raft_consensus.cc:348] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.603085 4465 raft_consensus.cc:374] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:51.603129 4465 raft_consensus.cc:729] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6346a24370614fa3a9fd748a834e41f3, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.603166 4465 consensus_queue.cc:260] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.603251 4465 ts_tablet_manager.cc:1428] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:51.605660 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.600149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40481 (local address 127.0.0.1:40932)
0925 14:47:51.600195 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.600205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.600238 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.602329 (+ 2091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.602330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.602332 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:51.602369 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.602371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.603545 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.603547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.604723 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.604726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.605433 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.605434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.605492 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.605517 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.605558 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.605600 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:47:51.605703 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.602011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40932 (local address 127.21.205.195:40481)
0925 14:47:51.602168 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.602172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.602182 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:47:51.602220 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.602222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.602233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.602396 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:51.602455 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.603440 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.604882 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.605352 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.605406 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.605463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.605585 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.605622 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":92,"threads_started":1}
I20260925 14:47:51.605741 4857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.600277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33992 (local address 127.21.205.194:39955)
0925 14:47:51.602213 (+ 1936us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.602218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.602229 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:47:51.602269 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.602271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.602287 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.602555 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:51.602627 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.603622 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.604952 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.605379 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.605507 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.605569 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.605620 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.605657 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1899,"thread_start_us":91,"threads_started":1}
I20260925 14:47:51.606009 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.600149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39955 (local address 127.0.0.1:33992)
0925 14:47:51.600413 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.600422 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.600473 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.602429 (+ 1956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.602430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.602433 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:51.602478 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.602480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.603773 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.603773 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.604824 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.604827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.605487 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.605488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.605553 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.605557 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.605606 (+ 49us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.605645 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":110,"thread_start_us":108,"threads_started":1}
I20260925 14:47:51.606364 4783 tablet_service.cc:1467] Processing CreateTablet for tablet acbe2a5e3a87402fae9ae791adf85c94 (DEFAULT_TABLE table=test-table [id=401f9c3818924eecbe20d5d3d9308fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:51.606679 4783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acbe2a5e3a87402fae9ae791adf85c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:51.606678 4649 tablet_service.cc:1467] Processing CreateTablet for tablet acbe2a5e3a87402fae9ae791adf85c94 (DEFAULT_TABLE table=test-table [id=401f9c3818924eecbe20d5d3d9308fc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:47:51.607033 4649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet acbe2a5e3a87402fae9ae791adf85c94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:47:51.609422 4860 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Bootstrap starting.
I20260925 14:47:51.609696 4861 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Bootstrap starting.
I20260925 14:47:51.610219 4860 tablet_bootstrap.cc:654] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:51.610373 4861 tablet_bootstrap.cc:654] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:47:51.610723 4860 log.cc:824] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:51.610848 4861 log.cc:824] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:51.611474 4860 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: No bootstrap required, opened a new log
I20260925 14:47:51.611527 4860 ts_tablet_manager.cc:1397] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:51.611625 4861 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: No bootstrap required, opened a new log
I20260925 14:47:51.611716 4861 ts_tablet_manager.cc:1397] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:47:51.612937 4860 raft_consensus.cc:348] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.613060 4860 raft_consensus.cc:374] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:51.613082 4860 raft_consensus.cc:729] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2271f634696a4fc0a5eb62c403df3367, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.613209 4860 consensus_queue.cc:260] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.613250 4861 raft_consensus.cc:348] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.613404 4861 raft_consensus.cc:374] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:47:51.613438 4861 raft_consensus.cc:729] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c97749581b44783ac1127112e6104ea, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.613497 4860 ts_tablet_manager.cc:1428] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:47:51.613571 4861 consensus_queue.cc:260] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.613639 4850 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
I20260925 14:47:51.613845 4861 ts_tablet_manager.cc:1428] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 14:47:51.613864 4716 heartbeater.cc:503] Master 127.21.205.254:34029 was elected leader, sending a full tablet report...
I20260925 14:47:51.673988 4865 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:51.674106 4865 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.674566 4865 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429)
I20260925 14:47:51.677407 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.674714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40481 (local address 127.21.205.194:36475)
0925 14:47:51.674793 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.674805 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.674847 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.674971 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.674972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.674973 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:51.675061 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.675063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.675742 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.675743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.676548 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.676551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.677134 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.677135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.677221 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.677225 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.677278 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.677308 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260925 14:47:51.677433 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.674808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:36475 (local address 127.21.205.195:40481)
0925 14:47:51.674846 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.674849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.674868 (+ 19us) server_negotiation.cc:407] Connection header received
0925 14:47:51.674895 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.674896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.674904 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.674960 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:51.675117 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.675670 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.676633 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.677054 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.677107 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.677114 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.677305 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.677352 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:47:51.677492 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.674784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:48621 (local address 127.21.205.193:36429)
0925 14:47:51.674835 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.674837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.674985 (+ 148us) server_negotiation.cc:407] Connection header received
0925 14:47:51.675083 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.675085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.675094 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.675146 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:47:51.675287 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.675756 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.676738 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.677131 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.677203 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.677211 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.677376 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.677420 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:47:51.677505 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.674716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:36429 (local address 127.21.205.194:48621)
0925 14:47:51.674900 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.674910 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:51.675009 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:51.675162 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:51.675163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:51.675165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:47:51.675226 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.675227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.675897 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.675897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.676626 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:51.676628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.677213 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.677214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:51.677293 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.677298 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:47:51.677352 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:47:51.677387 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":26,"thread_start_us":85,"threads_started":1}
I20260925 14:47:51.677816 4803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acbe2a5e3a87402fae9ae791adf85c94" candidate_uuid: "5c97749581b44783ac1127112e6104ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2271f634696a4fc0a5eb62c403df3367" is_pre_election: true
I20260925 14:47:51.677975 4803 raft_consensus.cc:2454] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c97749581b44783ac1127112e6104ea in term 0.
I20260925 14:47:51.677983 4528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acbe2a5e3a87402fae9ae791adf85c94" candidate_uuid: "5c97749581b44783ac1127112e6104ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6346a24370614fa3a9fd748a834e41f3" is_pre_election: true
I20260925 14:47:51.678131 4528 raft_consensus.cc:2454] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c97749581b44783ac1127112e6104ea in term 0.
I20260925 14:47:51.678193 4602 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea; no voters:
I20260925 14:47:51.678326 4865 raft_consensus.cc:2781] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:47:51.678455 4865 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:47:51.678488 4865 raft_consensus.cc:3037] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:51.679128 4865 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.679261 4865 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [CANDIDATE]: Term 1 election: Requested vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429)
I20260925 14:47:51.679394 4803 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acbe2a5e3a87402fae9ae791adf85c94" candidate_uuid: "5c97749581b44783ac1127112e6104ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2271f634696a4fc0a5eb62c403df3367"
I20260925 14:47:51.679420 4528 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "acbe2a5e3a87402fae9ae791adf85c94" candidate_uuid: "5c97749581b44783ac1127112e6104ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6346a24370614fa3a9fd748a834e41f3"
I20260925 14:47:51.679481 4803 raft_consensus.cc:3037] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:51.679486 4528 raft_consensus.cc:3037] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:47:51.680277 4803 raft_consensus.cc:2454] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c97749581b44783ac1127112e6104ea in term 1.
I20260925 14:47:51.680306 4528 raft_consensus.cc:2454] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c97749581b44783ac1127112e6104ea in term 1.
I20260925 14:47:51.680447 4602 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea; no voters:
I20260925 14:47:51.680557 4865 raft_consensus.cc:2781] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:47:51.680732 4865 raft_consensus.cc:686] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 1 LEADER]: Becoming Leader. State: Replica: 5c97749581b44783ac1127112e6104ea, State: Running, Role: LEADER
I20260925 14:47:51.680825 4865 consensus_queue.cc:237] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:51.681650 4383 catalog_manager.cc:5495] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c97749581b44783ac1127112e6104ea (127.21.205.194). New cstate: current_term: 1 leader_uuid: "5c97749581b44783ac1127112e6104ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } health_report { overall_health: UNKNOWN } } }
I20260925 14:47:51.697002 4857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.694980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34008 (local address 127.21.205.194:39955)
0925 14:47:51.695034 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:51.695036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:51.695045 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:47:51.695081 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:51.695082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:51.695096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:51.695152 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:47:51.695255 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.695673 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.696332 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:51.696510 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:51.696558 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:51.696682 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:47:51.696788 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:47:51.696845 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:51.696877 (+ 32us) server_negotiation.cc:299] Negotiation successful
0925 14:47:51.696893 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:47:51.698551 4803 raft_consensus.cc:1261] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Refusing update from remote peer 5c97749581b44783ac1127112e6104ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:47:51.698585 4528 raft_consensus.cc:1261] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Refusing update from remote peer 5c97749581b44783ac1127112e6104ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260925 14:47:51.698793 4717 tablet.cc:2369] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:47:51.698861 4867 consensus_queue.cc:1035] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:51.698944 4865 consensus_queue.cc:1035] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:47:51.701155 4871 mvcc.cc:205] Tried to move back new op lower bound from 7333264063275053056 to 7333264063205232640. Current Snapshot: MvccSnapshot[applied={T|T < 7333264063275053056}]
I20260925 14:47:51.701480 4872 mvcc.cc:205] Tried to move back new op lower bound from 7333264063275053056 to 7333264063205232640. Current Snapshot: MvccSnapshot[applied={T|T < 7333264063275053056}]
I20260925 14:47:51.703140 4383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:51.703235 4383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40626:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:47:51.703967 4383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Sending DeleteTablet for 3 replicas of tablet acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:51.704291 4649 tablet_service.cc:1514] Processing DeleteTablet for tablet acbe2a5e3a87402fae9ae791adf85c94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:33992
I20260925 14:47:51.704301 4783 tablet_service.cc:1514] Processing DeleteTablet for tablet acbe2a5e3a87402fae9ae791adf85c94 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:47:51 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:40932
I20260925 14:47:51.704527 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4422
I20260925 14:47:51.704535 4880 tablet_replica.cc:330] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: stopping tablet replica
I20260925 14:47:51.704556 4881 tablet_replica.cc:330] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: stopping tablet replica
I20260925 14:47:51.704682 4881 raft_consensus.cc:2229] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:47:51.704684 4880 raft_consensus.cc:2229] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:47:51.704802 4881 raft_consensus.cc:2258] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.705008 4880 raft_consensus.cc:2258] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:47:51.705411 4880 ts_tablet_manager.cc:1905] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:51.706939 4880 ts_tablet_manager.cc:1918] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:51.707023 4880 log.cc:1196] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:51.707278 4880 ts_tablet_manager.cc:1939] T acbe2a5e3a87402fae9ae791adf85c94 P 2271f634696a4fc0a5eb62c403df3367: Deleting consensus metadata
I20260925 14:47:51.707722 4370 catalog_manager.cc:4841] TS 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): tablet acbe2a5e3a87402fae9ae791adf85c94 (table test-table [id=401f9c3818924eecbe20d5d3d9308fc9]) successfully deleted
I20260925 14:47:51.709056 4881 ts_tablet_manager.cc:1905] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:47:51.710332 4881 ts_tablet_manager.cc:1918] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:47:51.710408 4881 log.cc:1196] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:51.710675 4881 ts_tablet_manager.cc:1939] T acbe2a5e3a87402fae9ae791adf85c94 P 5c97749581b44783ac1127112e6104ea: Deleting consensus metadata
I20260925 14:47:51.711054 4368 catalog_manager.cc:4841] TS 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): tablet acbe2a5e3a87402fae9ae791adf85c94 (table test-table [id=401f9c3818924eecbe20d5d3d9308fc9]) successfully deleted
W20260925 14:47:51.715392 4371 connection.cc:535] client connection to 127.21.205.193:36429 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 14:47:51.715478 4371 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 14:47:51.715860 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4580
I20260925 14:47:51.715951 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:51.715825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:36429 (local address 127.0.0.1:53278)
0925 14:47:51.715861 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:51.715900 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260925 14:47:51.717664 4371 catalog_manager.cc:4584] TS 6346a24370614fa3a9fd748a834e41f3 (127.21.205.193:36429): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet acbe2a5e3a87402fae9ae791adf85c94: Network error: Client connection negotiation failed: client connection to 127.21.205.193:36429: connect: Connection refused (error 111)
I20260925 14:47:51.721566 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4719
I20260925 14:47:51.725967 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4351
I20260925 14:47:51.731089 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--webserver_interface=127.21.205.254
--webserver_port=43125
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:34029
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:51.813938 4883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.814122 4883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.814143 4883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.815258 4883 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:47:51.815299 4883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.815312 4883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.815325 4883 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:47:51.815376 4883 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:47:51.816848 4883 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:34029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=43125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4883
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.817093 4883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.817318 4883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:51.821866 4889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.821983 4883 server_base.cc:1034] running on GCE node
W20260925 14:47:51.821913 4888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.821875 4891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.822314 4883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.822629 4883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.823868 4883 hybrid_clock.cc:648] HybridClock initialized: now 1790347671823839 us; error 42 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.826174 4883 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:47:51.826584 4883 webserver.cc:460] Webserver started at http://127.21.205.254:43125/ using document root <none> and password file <none>
I20260925 14:47:51.826817 4883 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.826891 4883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.828249 4883 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:51.828836 4898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.829016 4883 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:51.829115 4883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "1b7b8084f80f426a9dccae26976370be"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f87497ebcefaba3b9de5e41c836bb974"
server_key_iv: "986193a8d7b035ce6cafee6273bcbf2c"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.829452 4883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:51.858729 4883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:51.860988 4883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:51.861147 4883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:51.865022 4883 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:34029
I20260925 14:47:51.865056 4951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:34029 every 8 connection(s)
I20260925 14:47:51.865427 4883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:47:51.865773 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4883
I20260925 14:47:51.866145 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:36429
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=46261
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:51.867766 4952 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:47:51.868423 4952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap starting.
I20260925 14:47:51.876991 4952 log.cc:824] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:51.879068 4952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:47:51.879393 4952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be: Bootstrap complete.
I20260925 14:47:51.881642 4952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.881879 4952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Initialized, Role: FOLLOWER
I20260925 14:47:51.882076 4952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.882156 4952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:47:51.882208 4952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:47:51.882261 4952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 2 FOLLOWER]: Advancing to term 3
I20260925 14:47:51.883212 4952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.883323 4952 leader_election.cc:302] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b7b8084f80f426a9dccae26976370be; no voters:
I20260925 14:47:51.883567 4952 leader_election.cc:288] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 14:47:51.883602 4957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 3 FOLLOWER]: Leader election won for term 3
I20260925 14:47:51.883700 4957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [term 3 LEADER]: Becoming Leader. State: Replica: 1b7b8084f80f426a9dccae26976370be, State: Running, Role: LEADER
I20260925 14:47:51.883863 4952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:47:51.883816 4957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } }
I20260925 14:47:51.884091 4958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:51.884096 4959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7b8084f80f426a9dccae26976370be. Latest consensus state: current_term: 3 leader_uuid: "1b7b8084f80f426a9dccae26976370be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b7b8084f80f426a9dccae26976370be" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34029 } } }
I20260925 14:47:51.884305 4959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:51.884500 4958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7b8084f80f426a9dccae26976370be [sys.catalog]: This master's current role is: LEADER
I20260925 14:47:51.884647 4962 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:47:51.885402 4962 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:47:51.885504 4962 catalog_manager.cc:1259] Loaded cluster ID: 425217ae0ba24143bd6a9556cea62b07
I20260925 14:47:51.885533 4962 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 14:47:51.885893 4962 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffa0ae7e9ad google::LogMessage::Fail() at ??:0
@ 0x7ffa0ae8188a google::LogMessage::SendToLog() at ??:0
@ 0x7ffa0ae7e4de google::LogMessage::Flush() at ??:0
@ 0x7ffa0ae7fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffa12d9f0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ffa12da88b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffa0cd4c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffa0cd41954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffa0dba56db start_thread at ??:0
@ 0x7ffa0997c61f clone at ??:0
W20260925 14:47:51.955935 4955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:51.956133 4955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:51.956167 4955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:51.957276 4955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:51.957342 4955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:51.957399 4955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:47:51.958889 4955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:36429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=46261
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:51.959187 4955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:51.959447 4955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:47:51.964505 4955 server_base.cc:1034] running on GCE node
W20260925 14:47:51.964478 4990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.964589 4993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:51.964488 4991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:51.964969 4955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:51.965231 4955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:51.966383 4955 hybrid_clock.cc:648] HybridClock initialized: now 1790347671966353 us; error 44 us; skew 500 ppm
Sep 25 14:47:51 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:51.968973 4955 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:47:51.969482 4955 webserver.cc:460] Webserver started at http://127.21.205.193:46261/ using document root <none> and password file <none>
I20260925 14:47:51.969718 4955 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:51.969784 4955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:51.971063 4955 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:51.971726 5000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:51.971935 4955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:47:51.972075 4955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "6346a24370614fa3a9fd748a834e41f3"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "f2d19cc04a7e3159969a97b797801d04"
server_key_iv: "15f7e12197cdb883f719fcbe0ddb861c"
server_key_version: "encryptionkey@0"
I20260925 14:47:51.972527 4955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:52.014719 4955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:52.017151 4955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:52.017314 4955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:52.017597 4955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:52.018127 5007 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 14:47:52.020081 4955 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 14:47:52.020143 4955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 14:47:52.020192 4955 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 14:47:52.020797 4955 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 14:47:52.020870 4955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 14:47:52.020936 5007 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Bootstrap starting.
I20260925 14:47:52.028710 4955 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:36429
I20260925 14:47:52.028831 5116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:36429 every 8 connection(s)
I20260925 14:47:52.029121 4955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:47:52.031996 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 4955
I20260925 14:47:52.032276 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:39955
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=42835
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:47:52.032310 5007 log.cc:824] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Log is configured to *not* fsync() on all Append() calls
I20260925 14:47:52.034830 5007 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:47:52.035135 5007 tablet_bootstrap.cc:492] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Bootstrap complete.
I20260925 14:47:52.036406 5007 ts_tablet_manager.cc:1397] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260925 14:47:52.037813 5007 raft_consensus.cc:348] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:52.038138 5007 raft_consensus.cc:729] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6346a24370614fa3a9fd748a834e41f3, State: Initialized, Role: FOLLOWER
I20260925 14:47:52.038273 5007 consensus_queue.cc:260] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:52.038533 5007 ts_tablet_manager.cc:1428] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260925 14:47:52.110603 5119 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:47:52.029848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:41417)
0925 14:47:52.030148 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.030183 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:52.030633 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:52.110392 (+ 79759us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":121,"threads_started":1}
W20260925 14:47:52.110872 5117 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:34029 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:34029: Client connection negotiation failed: client connection to 127.21.205.254:34029: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 14:47:52.111246 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.111105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:59035)
0925 14:47:52.111164 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.111217 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 14:47:52.111582 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.111483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:53973)
0925 14:47:52.111532 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.111560 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:47:52.111708 5117 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:47:52.120839 5121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:52.121023 5121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:52.121045 5121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:52.122035 5121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:52.122066 5121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:52.122089 5121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:47:52.123407 5121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:39955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=42835
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:52.123634 5121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:52.123867 5121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:52.128537 5128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:52.128571 5131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:52.128544 5129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:52.128763 5121 server_base.cc:1034] running on GCE node
I20260925 14:47:52.128962 5121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:52.129196 5121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:52.130352 5121 hybrid_clock.cc:648] HybridClock initialized: now 1790347672130335 us; error 29 us; skew 500 ppm
Sep 25 14:47:52 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:52.132786 5121 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:47:52.133229 5121 webserver.cc:460] Webserver started at http://127.21.205.194:42835/ using document root <none> and password file <none>
I20260925 14:47:52.133445 5121 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:52.133510 5121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:52.134775 5121 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:52.135334 5138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.135501 5121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:52.135617 5121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "5c97749581b44783ac1127112e6104ea"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "e2156945ad73ffec25a2896c42dacec0"
server_key_iv: "7a9606cad94ed89713fdab6b9c8adfe4"
server_key_version: "encryptionkey@0"
I20260925 14:47:52.135999 5121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:52.161278 5121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:52.163540 5121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:52.163708 5121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:52.163961 5121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:52.164245 5121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:52.164297 5121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.164333 5121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:52.164361 5121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.169891 5121 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:39955
I20260925 14:47:52.169934 5252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:39955 every 8 connection(s)
I20260925 14:47:52.170259 5121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:47:52.171157 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.170833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:46295)
0925 14:47:52.170967 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.171035 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
W20260925 14:47:52.171530 5253 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:34029 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:34029: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
I20260925 14:47:52.171815 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.171705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:59197)
0925 14:47:52.171743 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.171766 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 14:47:52.172039 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.171988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:44909)
0925 14:47:52.172012 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.172027 (+ 15us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260925 14:47:52.172149 5253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 14:47:52.177408 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5121
I20260925 14:47:52.177702 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:40481
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:34029
--webserver_port=36717
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:47:52.255012 5256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:47:52.255225 5256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:47:52.255259 5256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:47:52.256358 5256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:47:52.256424 5256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:47:52.256469 5256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:47:52.257925 5256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:40481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=36717
--tserver_master_addrs=127.21.205.254:34029
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:47:52.258181 5256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:47:52.258430 5256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:47:52.263245 5261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:52.263247 5264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:47:52.263275 5262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:47:52.263367 5256 server_base.cc:1034] running on GCE node
I20260925 14:47:52.263752 5256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:47:52.263983 5256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:47:52.265152 5256 hybrid_clock.cc:648] HybridClock initialized: now 1790347672265134 us; error 31 us; skew 500 ppm
Sep 25 14:47:52 dist-test-slave-d7lw krb5kdc[3795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347672, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:47:52.267470 5256 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:47:52.267931 5256 webserver.cc:460] Webserver started at http://127.21.205.195:36717/ using document root <none> and password file <none>
I20260925 14:47:52.268108 5256 fs_manager.cc:359] Metadata directory not provided
I20260925 14:47:52.268146 5256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:47:52.269372 5256 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:47:52.269882 5271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.270035 5256 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:47:52.270108 5256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "2271f634696a4fc0a5eb62c403df3367"
format_stamp: "Formatted at 2026-09-25 14:47:50 on dist-test-slave-d7lw"
server_key: "2be6e522d8d6d33efd1a86f62e00d3ba"
server_key_iv: "3379d2ecdc9d2d6ce85811f7a8369f29"
server_key_version: "encryptionkey@0"
I20260925 14:47:52.270407 5256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:47:52.282529 5256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:47:52.284646 5256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:47:52.284747 5256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:47:52.284910 5256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:47:52.285128 5256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:47:52.285156 5256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.285175 5256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:47:52.285187 5256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:47:52.290555 5256 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:40481
I20260925 14:47:52.290621 5387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:40481 every 8 connection(s)
I20260925 14:47:52.290944 5256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:47:52.291975 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.291580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:55395)
0925 14:47:52.291717 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.291827 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260925 14:47:52.292389 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5256
W20260925 14:47:52.292410 5388 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:34029 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:34029: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
I20260925 14:47:52.292739 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.292619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:33389)
0925 14:47:52.292678 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.292713 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 14:47:52.293077 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:52.293009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:36413)
0925 14:47:52.293040 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:52.293062 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 14:47:52.293202 5388 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 14:47:53.112352 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.112141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:47403)
0925 14:47:53.112257 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:53.112291 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":56,"threads_started":1}
I20260925 14:47:53.172636 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.172411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:47759)
0925 14:47:53.172556 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:53.172588 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":66,"threads_started":1}
I20260925 14:47:53.293708 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.293527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:55619)
0925 14:47:53.293632 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:53.293663 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":51,"threads_started":1}
I20260925 14:47:53.768453 5396 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:53.768628 5396 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:53.769026 5396 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
Sep 25 14:47:53 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.194@KRBTEST.COM
Sep 25 14:47:53 dist-test-slave-d7lw krb5kdc[3795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347671, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.195@KRBTEST.COM
I20260925 14:47:53.779359 5398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.769291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:60321 (local address 127.21.205.194:39955)
0925 14:47:53.769474 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:53.769486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:53.769512 (+ 26us) server_negotiation.cc:407] Connection header received
0925 14:47:53.770018 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:53.770021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:53.770069 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:53.770172 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:53.770949 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.771855 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.772848 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.773252 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.773368 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:53.775619 (+ 2251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:53.775623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:53.775623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:53.775632 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:53.777677 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778154 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:53.778156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:53.778157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:53.778257 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778520 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:53.778521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:53.778522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:53.778754 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:53.778872 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:53.779125 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 14:47:53.779239 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":62,"threads_started":1}
I20260925 14:47:53.779337 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.769351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:40481 (local address 127.21.205.193:58193)
0925 14:47:53.769548 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:53.769559 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:53.769624 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:53.770253 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:53.770254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:53.770605 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:53.770801 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:53.770806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.771896 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.771897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:53.772755 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:53.772757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.773353 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.773355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:53.773452 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:53.774131 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:53.774136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:53.775939 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:53.777920 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:53.777921 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:53.777922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:53.778142 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:53.778396 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:53.778397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:53.778459 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:53.778837 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:53.778844 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:53.778970 (+ 126us) client_negotiation.cc:769] Sending connection context
0925 14:47:53.779066 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 14:47:53.779247 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":23,"thread_start_us":42,"threads_started":1}
I20260925 14:47:53.779390 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.769325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:58193 (local address 127.21.205.195:40481)
0925 14:47:53.769474 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 14:47:53.769486 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 14:47:53.769631 (+ 145us) server_negotiation.cc:407] Connection header received
0925 14:47:53.770074 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:47:53.770076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:47:53.770110 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:47:53.770247 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:47:53.770951 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.771764 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.772844 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.773250 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.773383 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:53.776044 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:47:53.776047 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:47:53.776047 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:47:53.776057 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:47:53.777844 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778220 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:53.778222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:53.778222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:53.778316 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778525 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:47:53.778527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:47:53.778527 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:47:53.778758 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:47:53.778836 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 14:47:53.779132 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 14:47:53.779272 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":57,"threads_started":1}
I20260925 14:47:53.779466 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:53.769189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:39955 (local address 127.21.205.193:60321)
0925 14:47:53.769454 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:53.769462 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:47:53.769509 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:47:53.770241 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:47:53.770243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:47:53.770603 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:47:53.770801 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:53.770806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.771937 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.771938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:53.772754 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:47:53.772757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:47:53.773339 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:47:53.773340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:47:53.773451 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:47:53.773992 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:47:53.774000 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:47:53.775519 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:47:53.777749 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:53.777756 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:53.777758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:53.778080 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:53.778340 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:47:53.778341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:47:53.778342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:47:53.778450 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:47:53.778860 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:47:53.778861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:47:53.778962 (+ 101us) client_negotiation.cc:769] Sending connection context
0925 14:47:53.779067 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 14:47:53.779323 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":156,"thread_start_us":72,"threads_started":1}
W20260925 14:47:53.780498 5001 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
W20260925 14:47:53.780591 5003 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:53.780643 5003 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6346a24370614fa3a9fd748a834e41f3; no voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea
I20260925 14:47:53.780712 5396 raft_consensus.cc:2726] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:47:54.112891 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:54.112746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:36389)
0925 14:47:54.112815 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:54.112850 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 14:47:54.173309 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:54.173117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:58929)
0925 14:47:54.173226 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:54.173268 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":55,"threads_started":1}
I20260925 14:47:54.294322 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:54.294067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:55039)
0925 14:47:54.294252 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:54.294286 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":51,"threads_started":1}
I20260925 14:47:55.113611 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:55.113356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:54011)
0925 14:47:55.113519 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:55.113562 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":61,"threads_started":1}
I20260925 14:47:55.174117 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:55.173891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:34935)
0925 14:47:55.174022 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:55.174068 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260925 14:47:55.295100 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:55.294862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:34013)
0925 14:47:55.295005 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:55.295049 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260925 14:47:55.635226 5406 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:55.635324 5406 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:55.635502 5406 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
W20260925 14:47:55.635952 5003 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
W20260925 14:47:55.636030 5001 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:55.636092 5001 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6346a24370614fa3a9fd748a834e41f3; no voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea
I20260925 14:47:55.636189 5406 raft_consensus.cc:2726] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:47:56.114490 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:56.114210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:59145)
0925 14:47:56.114377 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:56.114422 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260925 14:47:56.174847 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:56.174630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:58667)
0925 14:47:56.174751 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:56.174798 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":58,"threads_started":1}
I20260925 14:47:56.296064 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:56.295775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:36595)
0925 14:47:56.295977 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:56.296016 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260925 14:47:57.115566 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:57.115218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:36465)
0925 14:47:57.115452 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:57.115514 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":64,"threads_started":1}
I20260925 14:47:57.175693 5411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:57.175429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:49461)
0925 14:47:57.175571 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:57.175628 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260925 14:47:57.296792 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:57.296514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:44271)
0925 14:47:57.296693 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:57.296741 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":77,"threads_started":1}
I20260925 14:47:58.116407 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:58.116140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:58523)
0925 14:47:58.116295 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:58.116357 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260925 14:47:58.176553 5414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:58.176262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:44049)
0925 14:47:58.176434 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:58.176482 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260925 14:47:58.297634 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:58.297389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:37741)
0925 14:47:58.297540 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:58.297587 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260925 14:47:58.466477 5416 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:47:58.466589 5416 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:47:58.466792 5416 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
W20260925 14:47:58.467216 5003 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
W20260925 14:47:58.467315 5001 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:47:58.467414 5001 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6346a24370614fa3a9fd748a834e41f3; no voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea
I20260925 14:47:58.467478 5416 raft_consensus.cc:2726] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:47:59.117098 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:59.116885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:48465)
0925 14:47:59.117015 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:59.117049 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":68,"threads_started":1}
I20260925 14:47:59.177357 5418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:59.177062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:39083)
0925 14:47:59.177235 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:59.177291 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":62,"threads_started":1}
I20260925 14:47:59.298418 5419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:47:59.298161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:38091)
0925 14:47:59.298323 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:47:59.298367 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":78,"threads_started":1}
I20260925 14:48:00.117913 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:00.117652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:48559)
0925 14:48:00.117804 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:00.117862 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260925 14:48:00.178092 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:00.177864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:56235)
0925 14:48:00.178003 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:00.178044 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":76,"threads_started":1}
I20260925 14:48:00.299155 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:00.298938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:39003)
0925 14:48:00.299064 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:00.299106 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260925 14:48:01.118726 5423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:01.118481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:56909)
0925 14:48:01.118634 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:01.118677 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260925 14:48:01.178813 5424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:01.178565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:43769)
0925 14:48:01.178724 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:01.178764 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":80,"threads_started":1}
I20260925 14:48:01.299994 5425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:01.299653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:57465)
0925 14:48:01.299903 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:01.299945 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":88,"threads_started":1}
I20260925 14:48:02.119647 5426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:02.119330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:59045)
0925 14:48:02.119543 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:02.119596 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260925 14:48:02.179630 5427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:02.179381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:56279)
0925 14:48:02.179532 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:02.179580 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":76,"threads_started":1}
I20260925 14:48:02.196491 5428 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:02.196594 5428 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:48:02.196767 5428 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
W20260925 14:48:02.197168 5003 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
W20260925 14:48:02.197247 5001 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:48:02.197326 5001 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6346a24370614fa3a9fd748a834e41f3; no voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea
I20260925 14:48:02.197410 5428 raft_consensus.cc:2726] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:48:02.300715 5429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:02.300448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:37097)
0925 14:48:02.300610 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:02.300649 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260925 14:48:03.120448 5430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:03.120210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:58123)
0925 14:48:03.120354 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:03.120400 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260925 14:48:03.180549 5431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:03.180236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:36803)
0925 14:48:03.180453 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:03.180498 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":123,"threads_started":1}
I20260925 14:48:03.301537 5432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:03.301265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:58965)
0925 14:48:03.301430 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:03.301485 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":89,"threads_started":1}
I20260925 14:48:04.121145 5433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:04.120931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:47165)
0925 14:48:04.121055 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:04.121096 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260925 14:48:04.181277 5434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:04.181037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:58731)
0925 14:48:04.181190 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:04.181227 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260925 14:48:04.302426 5435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:04.302139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:59993)
0925 14:48:04.302327 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:04.302376 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":69,"threads_started":1}
I20260925 14:48:05.121975 5436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:05.121695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:60019)
0925 14:48:05.121881 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:05.121925 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
I20260925 14:48:05.182032 5437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:05.181783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:44679)
0925 14:48:05.181941 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:05.181983 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
I20260925 14:48:05.303067 5438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:05.302866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:46541)
0925 14:48:05.302985 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:05.303016 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260925 14:48:06.003701 5439 raft_consensus.cc:482] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:06.003857 5439 raft_consensus.cc:504] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2271f634696a4fc0a5eb62c403df3367" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 40481 } } peers { permanent_uuid: "5c97749581b44783ac1127112e6104ea" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 39955 } } peers { permanent_uuid: "6346a24370614fa3a9fd748a834e41f3" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 36429 } }
I20260925 14:48:06.004031 5439 leader_election.cc:288] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481), 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955)
W20260925 14:48:06.004441 5003 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2271f634696a4fc0a5eb62c403df3367 (127.21.205.195:40481): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
W20260925 14:48:06.004518 5001 leader_election.cc:341] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c97749581b44783ac1127112e6104ea (127.21.205.194:39955): Not found: Tablet not found: acbe2a5e3a87402fae9ae791adf85c94
I20260925 14:48:06.004563 5001 leader_election.cc:302] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6346a24370614fa3a9fd748a834e41f3; no voters: 2271f634696a4fc0a5eb62c403df3367, 5c97749581b44783ac1127112e6104ea
I20260925 14:48:06.004655 5439 raft_consensus.cc:2726] T acbe2a5e3a87402fae9ae791adf85c94 P 6346a24370614fa3a9fd748a834e41f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:48:06.122912 5440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:06.122677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:47671)
0925 14:48:06.122816 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:06.122867 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260925 14:48:06.182694 5441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:06.182492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:38525)
0925 14:48:06.182613 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:06.182645 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260925 14:48:06.303877 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:06.303574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:49891)
0925 14:48:06.303732 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:06.303817 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260925 14:48:07.123766 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:07.123543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.193:53623)
0925 14:48:07.123677 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:07.123718 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260925 14:48:07.183561 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:07.183340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.194:47113)
0925 14:48:07.183475 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:07.183512 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260925 14:48:07.298319 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 4955
I20260925 14:48:07.303730 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5121
I20260925 14:48:07.305574 5445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:07.305366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:34029 (local address 127.21.205.195:40131)
0925 14:48:07.305498 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:07.305529 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:34029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260925 14:48:07.308974 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5256
2026-09-25T14:48:07Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18505 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:48:07 dist-test-slave-d7lw krb5kdc[5448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:48:07 dist-test-slave-d7lw krb5kdc[5448](info): set up 2 sockets
Sep 25 14:48:07 dist-test-slave-d7lw krb5kdc[5448](info): commencing operation
krb5kdc: starting...
W20260925 14:48:08.355160 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:48:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:48:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:48:08.479737 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:08.565255 5461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:08.565485 5461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:08.565510 5461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:08.566772 5461 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 14:48:08.566826 5461 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 14:48:08.566847 5461 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 14:48:08.566860 5461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:48:08.566874 5461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:08.566898 5461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:08.566911 5461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:48:08.566978 5461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:48:08.568594 5461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:08.568902 5461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:08.569176 5461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:48:08.574667 5461 server_base.cc:1034] running on GCE node
W20260925 14:48:08.574625 5466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:08.574640 5467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:08.574625 5469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:08.575171 5461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:08.575528 5461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:08.576684 5461 hybrid_clock.cc:648] HybridClock initialized: now 1790347688576651 us; error 56 us; skew 500 ppm
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:08.579257 5461 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:48:08.579859 5461 webserver.cc:460] Webserver started at http://127.21.205.254:43753/ using document root <none> and password file <none>
I20260925 14:48:08.580139 5461 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:08.580216 5461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:08.580382 5461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:48:08.581521 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "869dcaae7d944adeabf285781dcf9246"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "206401f7a343d8bfddd28d33706fd73f"
server_key_iv: "81216471cc2de911b23d1eb6cc4b871e"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.581925 5461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "869dcaae7d944adeabf285781dcf9246"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "206401f7a343d8bfddd28d33706fd73f"
server_key_iv: "81216471cc2de911b23d1eb6cc4b871e"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.583333 5461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:48:08.584172 5477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.584353 5461 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.584475 5461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "869dcaae7d944adeabf285781dcf9246"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "206401f7a343d8bfddd28d33706fd73f"
server_key_iv: "81216471cc2de911b23d1eb6cc4b871e"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.584571 5461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:08.595598 5461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:08.597858 5461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:08.598052 5461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:08.602381 5461 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42575
I20260925 14:48:08.602424 5530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42575 every 8 connection(s)
I20260925 14:48:08.602831 5461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:48:08.603760 5531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:08.604974 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5461
I20260925 14:48:08.605077 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:48:08.605288 22327 external_mini_cluster.cc:1428] Setting key 0a4e2bdd8969f295f7f8a7195a45fd15
I20260925 14:48:08.606752 5531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap starting.
I20260925 14:48:08.607396 5531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:08.607715 5531 log.cc:824] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:08.608495 5531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: No bootstrap required, opened a new log
I20260925 14:48:08.610316 5531 raft_consensus.cc:348] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:08.610507 5531 raft_consensus.cc:374] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:08.610559 5531 raft_consensus.cc:729] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Initialized, Role: FOLLOWER
I20260925 14:48:08.610719 5531 consensus_queue.cc:260] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:08.610824 5531 raft_consensus.cc:388] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:48:08.610869 5531 raft_consensus.cc:482] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:48:08.610917 5531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:08.611497 5531 raft_consensus.cc:504] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:08.611624 5531 leader_election.cc:302] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 869dcaae7d944adeabf285781dcf9246; no voters:
I20260925 14:48:08.611860 5531 leader_election.cc:288] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:48:08.611912 5536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:48:08.612123 5536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 LEADER]: Becoming Leader. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Running, Role: LEADER
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:08.612205 5531 sys_catalog.cc:564] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:48:08.612275 5536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:08.612677 5538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:08.612816 5538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:08.613071 5545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:48:08.612670 5536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 869dcaae7d944adeabf285781dcf9246. Latest consensus state: current_term: 1 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:08.614020 5545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:48:08.614223 5536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:08.616092 5545 catalog_manager.cc:1347] Generated new cluster ID: c351fbd7f8d34918ac1dd1d2981920b1
I20260925 14:48:08.616149 5545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:48:08.616888 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:08.605896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55646 (local address 127.21.205.254:42575)
0925 14:48:08.606103 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:08.606113 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:08.606139 (+ 26us) server_negotiation.cc:407] Connection header received
0925 14:48:08.606563 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:08.606570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:08.606617 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:08.606801 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:08.607439 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.608531 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.609348 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.609786 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.609918 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:08.612807 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:08.612814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:08.612814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:08.612828 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:08.615016 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.615466 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.615469 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.615470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.615591 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.615866 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.615867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.615867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.616189 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:08.616326 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:08.616559 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 14:48:08.616707 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":82,"threads_started":1}
I20260925 14:48:08.634055 5545 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:48:08.634902 5545 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:48:08.640067 5545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Generated new TSK 0
I20260925 14:48:08.640323 5545 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:48:08.679375 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:08.785084 5557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:08.785310 5557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:08.785336 5557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:08.786549 5557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:08.786602 5557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:08.786661 5557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:48:08.788363 5557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:08.788640 5557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:08.788856 5557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:08.794440 5565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:08.794443 5563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:08.794549 5557 server_base.cc:1034] running on GCE node
W20260925 14:48:08.794440 5562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:08.794989 5557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:08.795305 5557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:08.796502 5557 hybrid_clock.cc:648] HybridClock initialized: now 1790347688796486 us; error 28 us; skew 500 ppm
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:08.799099 5557 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:48:08.799590 5557 webserver.cc:460] Webserver started at http://127.21.205.193:43675/ using document root <none> and password file <none>
I20260925 14:48:08.799901 5557 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:08.799978 5557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:08.800155 5557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:48:08.801254 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "5ae1dac631df44b086bce2f3e254bd3a"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "05d4768cb52fb2651405241150130633"
server_key_iv: "6605a2c88da68ffe5da02c9895daa36d"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.801647 5557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "5ae1dac631df44b086bce2f3e254bd3a"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "05d4768cb52fb2651405241150130633"
server_key_iv: "6605a2c88da68ffe5da02c9895daa36d"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.803035 5557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:48:08.803844 5572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.804044 5557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:08.804150 5557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "5ae1dac631df44b086bce2f3e254bd3a"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "05d4768cb52fb2651405241150130633"
server_key_iv: "6605a2c88da68ffe5da02c9895daa36d"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.804242 5557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:08.817401 5557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:08.819433 5557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:08.819640 5557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:08.819960 5557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:08.820300 5557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:08.820367 5557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.820410 5557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:08.820439 5557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.826364 5557 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37473
I20260925 14:48:08.826453 5686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37473 every 8 connection(s)
I20260925 14:48:08.826797 5557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:08.835884 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5557
I20260925 14:48:08.836016 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:48:08.836242 22327 external_mini_cluster.cc:1428] Setting key 2ffe5ca69f05984f3e2f0e3b7a392c19
I20260925 14:48:08.836757 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:08.827582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:43079 (local address 127.21.205.254:42575)
0925 14:48:08.827673 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:08.827676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:08.827939 (+ 263us) server_negotiation.cc:407] Connection header received
0925 14:48:08.828524 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:08.828525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:08.828538 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:08.828598 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:08.829712 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.830351 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.831398 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.831618 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.831702 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:08.833948 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:08.833949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:08.833950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:08.833959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:08.835107 (+ 1148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.835563 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.835565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.835565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.835623 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.835962 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.835964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.835964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.836127 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:08.836193 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:08.836568 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 14:48:08.836666 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:48:08.836992 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:08.827547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:43079)
0925 14:48:08.827736 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:08.827844 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:08.828406 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:08.828695 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:08.828698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:08.829392 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:08.829604 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:08.829609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.830455 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.830457 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:08.831292 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:08.831295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.831714 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.831716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:08.831865 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:08.832367 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:08.832376 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:08.833838 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:08.835214 (+ 1376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:08.835223 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:08.835225 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:08.835461 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:08.835694 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:08.835695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:08.835695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:08.835883 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:08.836183 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:08.836190 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:08.836384 (+ 194us) client_negotiation.cc:769] Sending connection context
0925 14:48:08.836535 (+ 151us) client_negotiation.cc:240] Negotiation successful
0925 14:48:08.836714 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260925 14:48:08.837463 5687 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:08.837589 5687 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:08.837813 5687 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:08.838400 5494 ts_manager.cc:194] Registered new tserver with Master: 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473)
I20260925 14:48:08.839370 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:43079
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:48:08.862749 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:08.946231 5692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:08.946461 5692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:08.946488 5692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:08.947664 5692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:08.947705 5692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:08.947731 5692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:48:08.949260 5692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:08.949481 5692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:08.949676 5692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:08.955144 5697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:08.955147 5698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:08.955144 5700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:08.955368 5692 server_base.cc:1034] running on GCE node
I20260925 14:48:08.955572 5692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:08.955924 5692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:08.957104 5692 hybrid_clock.cc:648] HybridClock initialized: now 1790347688957098 us; error 38 us; skew 500 ppm
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:08.963917 5692 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:48:08.964398 5692 webserver.cc:460] Webserver started at http://127.21.205.194:36033/ using document root <none> and password file <none>
I20260925 14:48:08.964651 5692 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:08.964727 5692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:08.964887 5692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:48:08.966039 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "870a3371a660461da0cc4ffd983cbfe0"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "fd3678e66aa7ecb3497c6c22051faceb"
server_key_iv: "9d9e3317f9886a00e33cecf26fa88771"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.966441 5692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "870a3371a660461da0cc4ffd983cbfe0"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "fd3678e66aa7ecb3497c6c22051faceb"
server_key_iv: "9d9e3317f9886a00e33cecf26fa88771"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.968052 5692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:48:08.968901 5707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.969115 5692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:48:08.969239 5692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "870a3371a660461da0cc4ffd983cbfe0"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "fd3678e66aa7ecb3497c6c22051faceb"
server_key_iv: "9d9e3317f9886a00e33cecf26fa88771"
server_key_version: "encryptionkey@0"
I20260925 14:48:08.969324 5692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:08.978732 5692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:08.981297 5692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:08.981479 5692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:08.981725 5692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:08.982050 5692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:08.982134 5692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.982170 5692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:08.982196 5692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:08.988237 5692 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38311
I20260925 14:48:08.988338 5824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38311 every 8 connection(s)
I20260925 14:48:08.988686 5692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:48:08 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347688, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:08.998025 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:08.989452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:36541 (local address 127.21.205.254:42575)
0925 14:48:08.989545 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:08.989547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:08.989749 (+ 202us) server_negotiation.cc:407] Connection header received
0925 14:48:08.990264 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:08.990265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:08.990279 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:08.990364 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:08.991326 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.991922 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.992951 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.993139 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.993191 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:08.995312 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:08.995313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:08.995314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:08.995321 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:08.996704 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.997105 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.997105 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.997106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.997155 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:08.997380 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:08.997380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:08.997381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:08.997487 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:08.997538 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:08.997891 (+ 353us) server_negotiation.cc:299] Negotiation successful
0925 14:48:08.997956 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 14:48:08.998167 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5692
I20260925 14:48:08.998272 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:48:08.998365 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:08.989415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:36541)
0925 14:48:08.989605 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:08.989643 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:08.990142 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:08.990419 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:08.990423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:08.991008 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:08.991219 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:08.991224 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.992012 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.992013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:08.992867 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:08.992870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:08.993216 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:08.993216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:08.993322 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:08.993802 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:08.993811 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:08.995218 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:08.996780 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:08.996787 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:08.996789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:08.997031 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:08.997216 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:08.997216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:08.997217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:08.997320 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:08.997576 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:08.997583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:08.997737 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 14:48:08.997869 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 14:48:08.998088 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260925 14:48:08.998523 22327 external_mini_cluster.cc:1428] Setting key d71c52cc408dc699635646082f3586c1
I20260925 14:48:08.999003 5825 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:08.999141 5825 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:08.999361 5825 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:08.999863 5494 ts_manager.cc:194] Registered new tserver with Master: 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311)
I20260925 14:48:09.000382 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:36541
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:48:09.026786 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:09.111281 5830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:09.111548 5830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:09.111589 5830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:09.113119 5830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:09.113193 5830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:09.113240 5830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:48:09.114933 5830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:09.115233 5830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:09.115496 5830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:09.120994 5836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:09.120983 5835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:09.120987 5838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:09.121342 5830 server_base.cc:1034] running on GCE node
I20260925 14:48:09.121557 5830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:09.121809 5830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:09.122967 5830 hybrid_clock.cc:648] HybridClock initialized: now 1790347689122943 us; error 39 us; skew 500 ppm
Sep 25 14:48:09 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:09.125663 5830 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:48:09.126102 5830 webserver.cc:460] Webserver started at http://127.21.205.195:33787/ using document root <none> and password file <none>
I20260925 14:48:09.126322 5830 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:09.126365 5830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:09.126461 5830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:48:09.127547 5830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
format_stamp: "Formatted at 2026-09-25 14:48:09 on dist-test-slave-d7lw"
server_key: "95e76fbee349595988da60ce8593f1be"
server_key_iv: "073f774cc4e80ad4939d59f762c3ef4c"
server_key_version: "encryptionkey@0"
I20260925 14:48:09.127945 5830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
format_stamp: "Formatted at 2026-09-25 14:48:09 on dist-test-slave-d7lw"
server_key: "95e76fbee349595988da60ce8593f1be"
server_key_iv: "073f774cc4e80ad4939d59f762c3ef4c"
server_key_version: "encryptionkey@0"
I20260925 14:48:09.129262 5830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:09.130046 5845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:09.130257 5830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:09.130384 5830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
format_stamp: "Formatted at 2026-09-25 14:48:09 on dist-test-slave-d7lw"
server_key: "95e76fbee349595988da60ce8593f1be"
server_key_iv: "073f774cc4e80ad4939d59f762c3ef4c"
server_key_version: "encryptionkey@0"
I20260925 14:48:09.130481 5830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:09.160392 5830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:09.162609 5830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:09.162789 5830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:09.163059 5830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:09.163404 5830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:09.163462 5830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:09.163506 5830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:09.163533 5830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:09.169615 5830 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36895
I20260925 14:48:09.169710 5960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36895 every 8 connection(s)
I20260925 14:48:09.170059 5830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:48:09.172551 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 5830
I20260925 14:48:09.172670 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:48:09.172892 22327 external_mini_cluster.cc:1428] Setting key bfcd4594c9637373a2f04ae4afb9db94
Sep 25 14:48:09 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347689, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:09.179766 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.170894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:35591 (local address 127.21.205.254:42575)
0925 14:48:09.170955 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.170957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.171150 (+ 193us) server_negotiation.cc:407] Connection header received
0925 14:48:09.171634 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.171635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.171645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.171705 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:09.172884 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.173580 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.174696 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.174898 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.174953 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.177273 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:09.177274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:09.177274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:09.177281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:09.178601 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:09.178973 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:09.178973 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:09.178974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:09.179023 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:09.179234 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:09.179234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:09.179234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:09.179339 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:09.179400 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.179638 (+ 238us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.179702 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:48:09.180069 5963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.170841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:35591)
0925 14:48:09.171014 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.171057 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.171550 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.171764 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.171769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.172511 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:09.172767 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.172775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.173662 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.173663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.174607 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.174610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.174978 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.174979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.175106 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.175755 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:09.175765 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:09.177187 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:09.178679 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:09.178686 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:09.178688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:09.178904 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:09.179083 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:09.179084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:09.179084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:09.179180 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:09.179407 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:09.179412 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:09.179532 (+ 120us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.179612 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.179735 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260925 14:48:09.180711 5961 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:09.180848 5961 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:09.181035 5961 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:09.181445 5494 ts_manager.cc:194] Registered new tserver with Master: 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
I20260925 14:48:09.181844 5494 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:35591
I20260925 14:48:09.184659 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:48:09.191922 5534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.186037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55654 (local address 127.21.205.254:42575)
0925 14:48:09.186092 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.186094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.186169 (+ 75us) server_negotiation.cc:407] Connection header received
0925 14:48:09.186206 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.186207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.186215 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.186268 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:09.186776 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.187336 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.188126 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.188368 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.188421 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.189540 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:09.189541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:09.189542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:09.189548 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:09.190919 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:09.191214 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:09.191215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:09.191215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:09.191263 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:09.191431 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:09.191431 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:09.191432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:09.191570 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:09.191646 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.191730 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.191852 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 14:48:09.193564 5494 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:48:09.195227 5494 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:48:09.202410 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.198263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40416 (local address 127.21.205.193:37473)
0925 14:48:09.198461 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.198465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.198570 (+ 105us) server_negotiation.cc:407] Connection header received
0925 14:48:09.198735 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.198737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.198751 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.198826 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:09.198958 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.200168 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.201449 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.201947 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202052 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202138 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.202242 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.202285 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260925 14:48:09.202451 5973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.197875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38311 (local address 127.0.0.1:60674)
0925 14:48:09.198050 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.198097 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.198166 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.198646 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.198649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.198651 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:09.198729 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.198731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.200314 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.200315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.201236 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.201239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.202147 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.202214 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202218 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.202248 (+ 30us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.202308 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":4,"thread_start_us":109,"threads_started":1}
I20260925 14:48:09.202433 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.198109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37473 (local address 127.0.0.1:40416)
0925 14:48:09.198486 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.198497 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.198636 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.198853 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.198854 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.198856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:09.198897 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.198898 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.200272 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.200273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.201369 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.201371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.202052 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.202132 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202147 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.202218 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.202288 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260925 14:48:09.202616 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.198012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36010 (local address 127.21.205.195:36895)
0925 14:48:09.198250 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.198255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.198437 (+ 182us) server_negotiation.cc:407] Connection header received
0925 14:48:09.198549 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.198551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.198566 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.198646 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:09.199046 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.199731 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.201371 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.202045 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202120 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202174 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.202475 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.202524 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260925 14:48:09.202679 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.197875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36895 (local address 127.0.0.1:36010)
0925 14:48:09.198343 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.198351 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.198467 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.198961 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.198962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.198963 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:09.198991 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.198992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.199900 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.199901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.201246 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.201248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.202201 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.202259 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202262 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.202305 (+ 43us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.202353 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":24,"thread_start_us":93,"threads_started":1}
I20260925 14:48:09.202899 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.198308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.21.205.194:38311)
0925 14:48:09.198479 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.198484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.198497 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:48:09.198541 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.198543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.198561 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.198642 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:09.198792 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.200169 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.201324 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.202045 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.202673 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.202735 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.202762 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.202813 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260925 14:48:09.203600 5894 tablet_service.cc:1467] Processing CreateTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f (DEFAULT_TABLE table=test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:09.203627 5758 tablet_service.cc:1467] Processing CreateTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f (DEFAULT_TABLE table=test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:09.204017 5758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84b90cc274f4fe78bb0a6be057b7b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:09.204023 5894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84b90cc274f4fe78bb0a6be057b7b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:09.206712 5978 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Bootstrap starting.
I20260925 14:48:09.206801 5979 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Bootstrap starting.
I20260925 14:48:09.207497 5978 tablet_bootstrap.cc:654] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:09.207525 5979 tablet_bootstrap.cc:654] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:09.207772 5621 tablet_service.cc:1467] Processing CreateTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f (DEFAULT_TABLE table=test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:09.208192 5978 log.cc:824] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:09.208231 5621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c84b90cc274f4fe78bb0a6be057b7b3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:09.208221 5979 log.cc:824] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:09.209018 5978 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: No bootstrap required, opened a new log
I20260925 14:48:09.209110 5978 ts_tablet_manager.cc:1397] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 14:48:09.209240 5979 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: No bootstrap required, opened a new log
I20260925 14:48:09.209316 5979 ts_tablet_manager.cc:1397] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 14:48:09.210749 5978 raft_consensus.cc:348] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.210942 5978 raft_consensus.cc:374] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:09.211005 5978 raft_consensus.cc:729] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e2c494548a49fdbb0e1ec7a1b06414, State: Initialized, Role: FOLLOWER
I20260925 14:48:09.211151 5978 consensus_queue.cc:260] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.211462 5978 ts_tablet_manager.cc:1428] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 14:48:09.211440 5979 raft_consensus.cc:348] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.211623 5961 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
I20260925 14:48:09.211705 5979 raft_consensus.cc:374] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:09.211846 5979 raft_consensus.cc:729] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870a3371a660461da0cc4ffd983cbfe0, State: Initialized, Role: FOLLOWER
I20260925 14:48:09.211953 5983 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap starting.
I20260925 14:48:09.212029 5979 consensus_queue.cc:260] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.212378 5979 ts_tablet_manager.cc:1428] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 14:48:09.212455 5825 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
I20260925 14:48:09.212709 5983 tablet_bootstrap.cc:654] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:09.213358 5983 log.cc:824] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:09.214326 5983 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: No bootstrap required, opened a new log
I20260925 14:48:09.214393 5983 ts_tablet_manager.cc:1397] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 14:48:09.215737 5983 raft_consensus.cc:348] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.215957 5983 raft_consensus.cc:374] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:09.215986 5983 raft_consensus.cc:729] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1dac631df44b086bce2f3e254bd3a, State: Initialized, Role: FOLLOWER
I20260925 14:48:09.216068 5983 consensus_queue.cc:260] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.216272 5983 ts_tablet_manager.cc:1428] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:48:09.216325 5687 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
W20260925 14:48:09.239358 5826 tablet.cc:2369] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:48:09.327756 5688 tablet.cc:2369] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:48:09.420671 5962 tablet.cc:2369] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:48:09.469028 5984 raft_consensus.cc:482] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:09.469163 5984 raft_consensus.cc:504] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.469619 5984 leader_election.cc:288] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895), 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473)
I20260925 14:48:09.473814 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.469948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:60707 (local address 127.21.205.195:36895)
0925 14:48:09.469996 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.469998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.470009 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:48:09.470079 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.470081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.470092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.470161 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:09.470384 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.471160 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.472826 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.473432 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.473496 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.473503 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.473698 (+ 195us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.473738 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:48:09.473829 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.470056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:40455 (local address 127.21.205.193:37473)
0925 14:48:09.470121 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.470123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.470201 (+ 78us) server_negotiation.cc:407] Connection header received
0925 14:48:09.470232 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.470233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.470249 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.470293 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:09.470457 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.471160 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.472833 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.473405 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.473489 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.473499 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.473695 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.473738 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:48:09.473826 5827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.469828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36895 (local address 127.21.205.194:60707)
0925 14:48:09.469913 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.469930 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.470007 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.470184 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.470185 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.470187 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:09.470271 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.470273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.471270 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.471270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.472722 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.472726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.473512 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.473513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.473607 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.473609 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.473671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.473730 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":108}
I20260925 14:48:09.473846 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.469988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37473 (local address 127.21.205.194:40455)
0925 14:48:09.470103 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.470111 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:09.470183 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:09.470321 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:09.470321 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:09.470323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:09.470388 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.470389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.471270 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.471270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.472722 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:09.472726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.473513 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.473513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:09.473604 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.473609 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:48:09.473671 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 14:48:09.473730 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":81,"thread_start_us":63,"threads_started":1}
I20260925 14:48:09.474282 5641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84b90cc274f4fe78bb0a6be057b7b3f" candidate_uuid: "870a3371a660461da0cc4ffd983cbfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1dac631df44b086bce2f3e254bd3a" is_pre_election: true
I20260925 14:48:09.474278 5915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84b90cc274f4fe78bb0a6be057b7b3f" candidate_uuid: "870a3371a660461da0cc4ffd983cbfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" is_pre_election: true
I20260925 14:48:09.474459 5641 raft_consensus.cc:2454] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 870a3371a660461da0cc4ffd983cbfe0 in term 0.
I20260925 14:48:09.474476 5915 raft_consensus.cc:2454] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 870a3371a660461da0cc4ffd983cbfe0 in term 0.
I20260925 14:48:09.474730 5711 leader_election.cc:302] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414; no voters:
I20260925 14:48:09.474901 5984 raft_consensus.cc:2781] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:48:09.474999 5984 raft_consensus.cc:482] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:48:09.475036 5984 raft_consensus.cc:3037] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:09.475801 5984 raft_consensus.cc:504] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.475967 5984 leader_election.cc:288] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [CANDIDATE]: Term 1 election: Requested vote from peers 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895), 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473)
I20260925 14:48:09.476172 5915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84b90cc274f4fe78bb0a6be057b7b3f" candidate_uuid: "870a3371a660461da0cc4ffd983cbfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
I20260925 14:48:09.476179 5641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c84b90cc274f4fe78bb0a6be057b7b3f" candidate_uuid: "870a3371a660461da0cc4ffd983cbfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ae1dac631df44b086bce2f3e254bd3a"
I20260925 14:48:09.476246 5915 raft_consensus.cc:3037] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:09.476274 5641 raft_consensus.cc:3037] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:09.476972 5915 raft_consensus.cc:2454] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 870a3371a660461da0cc4ffd983cbfe0 in term 1.
I20260925 14:48:09.476996 5641 raft_consensus.cc:2454] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 870a3371a660461da0cc4ffd983cbfe0 in term 1.
I20260925 14:48:09.477152 5711 leader_election.cc:302] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414; no voters:
I20260925 14:48:09.477295 5984 raft_consensus.cc:2781] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:48:09.477504 5984 raft_consensus.cc:686] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 870a3371a660461da0cc4ffd983cbfe0, State: Running, Role: LEADER
I20260925 14:48:09.477612 5984 consensus_queue.cc:237] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:09.478488 5494 catalog_manager.cc:5495] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "870a3371a660461da0cc4ffd983cbfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } health_report { overall_health: HEALTHY } } }
I20260925 14:48:09.485924 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.483234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.21.205.194:38311)
0925 14:48:09.483300 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:09.483302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:09.483312 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:48:09.483402 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:09.483403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:09.483425 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:09.483512 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:48:09.483648 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.484205 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.485042 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:09.485330 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:09.485406 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:09.485573 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:48:09.485690 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:48:09.485789 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:09.485811 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 14:48:09.485835 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:48:09.488193 5915 raft_consensus.cc:1261] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Refusing update from remote peer 870a3371a660461da0cc4ffd983cbfe0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:48:09.488193 5641 raft_consensus.cc:1261] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Refusing update from remote peer 870a3371a660461da0cc4ffd983cbfe0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:48:09.488595 5984 consensus_queue.cc:1035] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:48:09.488867 5988 consensus_queue.cc:1035] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:48:09.491873 5992 mvcc.cc:205] Tried to move back new op lower bound from 7333264136140173312 to 7333264136100954112. Current Snapshot: MvccSnapshot[applied={T|T < 7333264136140173312}]
I20260925 14:48:09.492127 5993 mvcc.cc:205] Tried to move back new op lower bound from 7333264136140173312 to 7333264136100954112. Current Snapshot: MvccSnapshot[applied={T|T < 7333264136140173312}]
I20260925 14:48:09.493626 5494 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:48:09.493719 5494 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:48:09.494547 5494 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Sending DeleteTablet for 3 replicas of tablet c84b90cc274f4fe78bb0a6be057b7b3f
I20260925 14:48:09.494900 5758 tablet_service.cc:1514] Processing DeleteTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:09 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:60674
I20260925 14:48:09.494906 5894 tablet_service.cc:1514] Processing DeleteTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:09 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:36010
I20260925 14:48:09.494952 5621 tablet_service.cc:1514] Processing DeleteTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:09 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:40416
I20260925 14:48:09.495251 6003 tablet_replica.cc:330] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: stopping tablet replica
I20260925 14:48:09.495338 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5557
I20260925 14:48:09.495363 6003 raft_consensus.cc:2229] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:48:09.495472 6003 raft_consensus.cc:2258] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:09.495244 6002 tablet_replica.cc:330] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: stopping tablet replica
I20260925 14:48:09.500722 6002 raft_consensus.cc:2229] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:48:09.500834 6002 raft_consensus.cc:2258] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:09.501019 6001 tablet_replica.cc:330] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: stopping tablet replica
I20260925 14:48:09.501134 6001 raft_consensus.cc:2229] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:48:09.501255 6001 raft_consensus.cc:2258] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:09.501446 6002 ts_tablet_manager.cc:1905] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:09.501829 6001 ts_tablet_manager.cc:1905] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:09.503753 6002 ts_tablet_manager.cc:1918] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:48:09.503778 6001 ts_tablet_manager.cc:1918] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:48:09.503873 6001 log.cc:1196] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/c84b90cc274f4fe78bb0a6be057b7b3f
I20260925 14:48:09.503983 6002 log.cc:1196] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/c84b90cc274f4fe78bb0a6be057b7b3f
I20260925 14:48:09.504200 6001 ts_tablet_manager.cc:1939] T c84b90cc274f4fe78bb0a6be057b7b3f P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting consensus metadata
I20260925 14:48:09.504369 6002 ts_tablet_manager.cc:1939] T c84b90cc274f4fe78bb0a6be057b7b3f P 870a3371a660461da0cc4ffd983cbfe0: Deleting consensus metadata
I20260925 14:48:09.504838 5480 catalog_manager.cc:4841] TS 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): tablet c84b90cc274f4fe78bb0a6be057b7b3f (table test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]) successfully deleted
W20260925 14:48:09.505021 5480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:37473 (error 108)
I20260925 14:48:09.504840 5479 catalog_manager.cc:4841] TS 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): tablet c84b90cc274f4fe78bb0a6be057b7b3f (table test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]) successfully deleted
I20260925 14:48:09.505220 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5692
I20260925 14:48:09.506274 5975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:09.506149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37473 (local address 127.0.0.1:40418)
0925 14:48:09.506199 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:09.506239 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 14:48:09.506490 5480 catalog_manager.cc:4584] TS 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c84b90cc274f4fe78bb0a6be057b7b3f: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37473: connect: Connection refused (error 111)
I20260925 14:48:09.511941 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5830
I20260925 14:48:09.517269 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 5461
I20260925 14:48:29.523273 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--webserver_interface=127.21.205.254
--webserver_port=43753
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:29.608306 6005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:29.608531 6005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:29.608556 6005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:29.609773 6005 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 14:48:29.609834 6005 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 14:48:29.609849 6005 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 14:48:29.609861 6005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:48:29.609874 6005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:29.609887 6005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:29.609911 6005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:48:29.609925 6005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:48:29.612159 6005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:29.612419 6005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:29.612633 6005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:29.621938 6012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.621958 6011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.621965 6014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:29.621990 6005 server_base.cc:1034] running on GCE node
I20260925 14:48:29.622396 6005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:29.622767 6005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:29.623906 6005 hybrid_clock.cc:648] HybridClock initialized: now 1790347709623896 us; error 34 us; skew 500 ppm
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:29.629716 6005 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:48:29.630276 6005 webserver.cc:460] Webserver started at http://127.21.205.254:43753/ using document root <none> and password file <none>
I20260925 14:48:29.630563 6005 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:29.630625 6005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:29.632195 6005 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:48:29.632943 6022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:29.633160 6005 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:29.633285 6005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "869dcaae7d944adeabf285781dcf9246"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "206401f7a343d8bfddd28d33706fd73f"
server_key_iv: "81216471cc2de911b23d1eb6cc4b871e"
server_key_version: "encryptionkey@0"
I20260925 14:48:29.633751 6005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:29.648393 6005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:29.650513 6005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:29.650722 6005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:29.655594 6005 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42575
I20260925 14:48:29.655629 6075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42575 every 8 connection(s)
I20260925 14:48:29.656147 6005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:48:29.657737 6076 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:48:29.658485 6076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap starting.
I20260925 14:48:29.659373 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6005
I20260925 14:48:29.659700 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:37473
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=43675
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:48:29.667557 6076 log.cc:824] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:29.669703 6076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:48:29.670061 6076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap complete.
I20260925 14:48:29.672593 6076 raft_consensus.cc:348] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:29.672876 6076 raft_consensus.cc:729] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Initialized, Role: FOLLOWER
I20260925 14:48:29.673095 6076 consensus_queue.cc:260] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:29.673197 6076 raft_consensus.cc:388] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:48:29.673255 6076 raft_consensus.cc:482] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:48:29.673314 6076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 1 FOLLOWER]: Advancing to term 2
I20260925 14:48:29.674414 6076 raft_consensus.cc:504] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:29.674549 6076 leader_election.cc:302] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 869dcaae7d944adeabf285781dcf9246; no voters:
I20260925 14:48:29.674768 6076 leader_election.cc:288] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 14:48:29.674817 6080 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 14:48:29.674981 6080 raft_consensus.cc:686] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 LEADER]: Becoming Leader. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Running, Role: LEADER
I20260925 14:48:29.675069 6076 sys_catalog.cc:564] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:48:29.675137 6080 consensus_queue.cc:237] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:29.675437 6080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 869dcaae7d944adeabf285781dcf9246. Latest consensus state: current_term: 2 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:29.675530 6080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:29.675446 6081 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:29.675697 6081 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:29.677060 6097 catalog_manager.cc:1259] Loaded cluster ID: c351fbd7f8d34918ac1dd1d2981920b1
I20260925 14:48:29.677120 6097 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: loading cluster ID for follower catalog manager: success
I20260925 14:48:29.678293 6097 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: acquiring CA information for follower catalog manager: success
I20260925 14:48:29.678759 6097 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:48:29.678857 6086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:48:29.679216 6086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:48:29.679330 6086 catalog_manager.cc:1259] Loaded cluster ID: c351fbd7f8d34918ac1dd1d2981920b1
I20260925 14:48:29.679363 6086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:48:29.679518 6086 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:48:29.679613 6086 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Loaded TSK: 0
I20260925 14:48:29.701524 6086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Generated new TSK 1
I20260925 14:48:29.701660 6086 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:48:29.750644 6078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:29.750962 6078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:29.751011 6078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:29.752625 6078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:29.752712 6078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:29.752771 6078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:48:29.754390 6078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:37473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=43675
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:29.754696 6078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:29.754977 6078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:29.761477 6105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.761366 6107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.761605 6104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:29.761945 6078 server_base.cc:1034] running on GCE node
I20260925 14:48:29.762159 6078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:29.762450 6078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:29.763613 6078 hybrid_clock.cc:648] HybridClock initialized: now 1790347709763594 us; error 37 us; skew 500 ppm
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:29.766363 6078 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:48:29.766943 6078 webserver.cc:460] Webserver started at http://127.21.205.193:43675/ using document root <none> and password file <none>
I20260925 14:48:29.767201 6078 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:29.767264 6078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:29.768842 6078 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:48:29.769579 6114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:29.769801 6078 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:48:29.769910 6078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "5ae1dac631df44b086bce2f3e254bd3a"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "05d4768cb52fb2651405241150130633"
server_key_iv: "6605a2c88da68ffe5da02c9895daa36d"
server_key_version: "encryptionkey@0"
I20260925 14:48:29.770301 6078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:29.784485 6078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:29.786924 6078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:29.787102 6078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:29.787387 6078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:29.788036 6121 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 14:48:29.790124 6078 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 14:48:29.790181 6078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 14:48:29.790210 6078 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 14:48:29.790951 6078 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 14:48:29.791014 6078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 14:48:29.791107 6121 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap starting.
I20260925 14:48:29.798637 6078 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37473
I20260925 14:48:29.798775 6230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37473 every 8 connection(s)
I20260925 14:48:29.799315 6078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:48:29.802798 6121 log.cc:824] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:29.805349 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6078
I20260925 14:48:29.805539 6121 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:48:29.805682 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:38311
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=36033
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:48:29.805841 6121 tablet_bootstrap.cc:492] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap complete.
I20260925 14:48:29.807054 6121 ts_tablet_manager.cc:1397] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.009s
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:29.808743 6121 raft_consensus.cc:348] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:29.814495 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.800285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:36109 (local address 127.21.205.254:42575)
0925 14:48:29.801679 (+ 1394us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:29.801691 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:29.801739 (+ 48us) server_negotiation.cc:407] Connection header received
0925 14:48:29.801902 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:29.801911 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:29.801975 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:29.802144 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:29.803037 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.804473 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.806012 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.806631 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.808816 (+ 2185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.809625 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:29.809630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:29.809630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:29.809643 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:29.812168 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.812756 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.812760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.812761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.812899 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.813215 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.813216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.813216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.813586 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:29.813690 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:29.814048 (+ 358us) server_negotiation.cc:299] Negotiation successful
0925 14:48:29.814214 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1322,"thread_start_us":86,"threads_started":1}
I20260925 14:48:29.814394 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.800163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:36109)
0925 14:48:29.800506 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:29.800556 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:29.801409 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:29.802221 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:29.802226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:29.802601 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:29.802882 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.802889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.804621 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.804622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.805899 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.805903 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.806753 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.806754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.806916 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.807600 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:29.807610 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:29.809483 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:29.812289 (+ 2806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.812299 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.812302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.812644 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.812990 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.812991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.812991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.813136 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.813672 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:29.813679 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:29.813842 (+ 163us) client_negotiation.cc:769] Sending connection context
0925 14:48:29.813958 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 14:48:29.814141 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260925 14:48:29.819890 6121 raft_consensus.cc:729] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1dac631df44b086bce2f3e254bd3a, State: Initialized, Role: FOLLOWER
I20260925 14:48:29.820171 6121 consensus_queue.cc:260] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } } peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } }
I20260925 14:48:29.820526 6121 ts_tablet_manager.cc:1428] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Time spent starting tablet: real 0.013s user 0.003s sys 0.000s
I20260925 14:48:29.820750 6231 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:29.820860 6231 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:29.821069 6231 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:29.822002 6039 ts_manager.cc:194] Registered new tserver with Master: 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473)
I20260925 14:48:29.823349 6039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:36109
I20260925 14:48:29.824611 6231 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.193@KRBTEST.COM
I20260925 14:48:29.833551 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.822717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37473 (local address 127.0.0.1:35120)
0925 14:48:29.823037 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:29.823076 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:29.823162 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:29.823672 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:29.823679 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:29.824064 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:29.824138 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.824141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.825684 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.825685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.826745 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.826749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.827458 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.827459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.827549 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.828207 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:29.828214 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:29.830051 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:29.832294 (+ 2243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.832300 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.832302 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.832608 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.832855 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.832856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.832856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.832915 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.833217 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:29.833224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:29.833283 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 14:48:29.833347 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 14:48:29.833476 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":100,"threads_started":1}
I20260925 14:48:29.833587 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.823304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.21.205.193:37473)
0925 14:48:29.823493 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:29.823498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:29.823509 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:48:29.823549 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:29.823550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:29.823566 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:29.824224 (+ 658us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:29.824493 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.825586 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.826856 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.827364 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.827439 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.830169 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:29.830183 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:29.830183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:29.830194 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:29.832177 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.832703 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.832707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.832707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.832784 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.832974 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.832975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.832975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.833153 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:29.833235 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:29.833406 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 14:48:29.833493 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260925 14:48:29.834045 6164 tablet_service.cc:1514] Processing DeleteTablet for tablet c84b90cc274f4fe78bb0a6be057b7b3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:09 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:35120
I20260925 14:48:29.834695 6240 tablet_replica.cc:330] stopping tablet replica
I20260925 14:48:29.834815 6240 raft_consensus.cc:2229] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:48:29.834892 6240 raft_consensus.cc:2258] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:29.835234 6240 ts_tablet_manager.cc:1905] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:29.837302 6240 ts_tablet_manager.cc:1918] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:48:29.837397 6240 log.cc:1196] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal/wals/c84b90cc274f4fe78bb0a6be057b7b3f
I20260925 14:48:29.837630 6240 ts_tablet_manager.cc:1939] T c84b90cc274f4fe78bb0a6be057b7b3f P 5ae1dac631df44b086bce2f3e254bd3a: Deleting consensus metadata
I20260925 14:48:29.838177 6025 catalog_manager.cc:4841] TS 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473): tablet c84b90cc274f4fe78bb0a6be057b7b3f (table test-table [id=f0cbc0a1093d426db3eddc82692ce2c5]) successfully deleted
W20260925 14:48:29.899014 6236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:29.899248 6236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:29.899282 6236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:29.900612 6236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:29.900664 6236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:29.900694 6236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:48:29.902287 6236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:38311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=36033
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:29.902593 6236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:29.902854 6236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:29.908160 6246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.908639 6249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:29.908751 6247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:29.908913 6236 server_base.cc:1034] running on GCE node
I20260925 14:48:29.909192 6236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:29.909483 6236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:29.910660 6236 hybrid_clock.cc:648] HybridClock initialized: now 1790347709910653 us; error 39 us; skew 500 ppm
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:29.913383 6236 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:48:29.913924 6236 webserver.cc:460] Webserver started at http://127.21.205.194:36033/ using document root <none> and password file <none>
I20260925 14:48:29.914170 6236 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:29.914269 6236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:29.915830 6236 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:48:29.916495 6256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:29.916690 6236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:48:29.916805 6236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "870a3371a660461da0cc4ffd983cbfe0"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "fd3678e66aa7ecb3497c6c22051faceb"
server_key_iv: "9d9e3317f9886a00e33cecf26fa88771"
server_key_version: "encryptionkey@0"
I20260925 14:48:29.917197 6236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:29.936969 6236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:29.939077 6236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:29.939265 6236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:29.939569 6236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:29.939980 6236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:29.940034 6236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:29.940071 6236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:29.940099 6236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:29.946718 6236 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38311
I20260925 14:48:29.946774 6370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38311 every 8 connection(s)
I20260925 14:48:29.947145 6236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:48:29.952843 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6236
I20260925 14:48:29.953200 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:36895
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=33787
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 14:48:29 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347709, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:29.957464 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.947915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:51901 (local address 127.21.205.254:42575)
0925 14:48:29.947979 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:29.947981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:29.948219 (+ 238us) server_negotiation.cc:407] Connection header received
0925 14:48:29.948702 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:29.948703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:29.948714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:29.948774 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:29.949737 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.950262 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.951286 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.951488 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.951540 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.954195 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:29.954197 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:29.954198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:29.954207 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:29.955727 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.956250 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.956252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.956252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.956321 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:29.956691 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:29.956692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:29.956693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:29.956887 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:29.956970 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:29.957255 (+ 285us) server_negotiation.cc:299] Negotiation successful
0925 14:48:29.957363 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:48:29.957674 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:29.947925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:51901)
0925 14:48:29.948083 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:29.948123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:29.948617 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:29.948840 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:29.948844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:29.949444 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:29.949653 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.949658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.950353 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.950353 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.951199 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:29.951201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:29.951566 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:29.951566 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:29.951679 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:29.952318 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:29.952327 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:29.954082 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:29.955862 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.955877 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.955880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.956160 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.956393 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:29.956394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:29.956394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:29.956595 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:29.956964 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:29.956970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:29.957108 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 14:48:29.957226 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 14:48:29.957418 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260925 14:48:29.958344 6371 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:29.958479 6371 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:29.958730 6371 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:29.959174 6039 ts_manager.cc:194] Registered new tserver with Master: 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311)
I20260925 14:48:29.959573 6039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:51901
W20260925 14:48:30.040349 6374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:30.040575 6374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:30.040601 6374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:30.041849 6374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:30.041904 6374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:30.041934 6374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:48:30.043541 6374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:36895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=33787
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:30.043845 6374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:30.044108 6374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:30.049463 6380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:30.049467 6382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:30.049467 6379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:30.049798 6374 server_base.cc:1034] running on GCE node
I20260925 14:48:30.050017 6374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:30.050328 6374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:30.051529 6374 hybrid_clock.cc:648] HybridClock initialized: now 1790347710051505 us; error 29 us; skew 500 ppm
Sep 25 14:48:30 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:30.054255 6374 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:48:30.054692 6374 webserver.cc:460] Webserver started at http://127.21.205.195:33787/ using document root <none> and password file <none>
I20260925 14:48:30.054889 6374 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:30.054943 6374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:30.056461 6374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:48:30.057060 6390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:30.057256 6374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:48:30.057370 6374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
format_stamp: "Formatted at 2026-09-25 14:48:09 on dist-test-slave-d7lw"
server_key: "95e76fbee349595988da60ce8593f1be"
server_key_iv: "073f774cc4e80ad4939d59f762c3ef4c"
server_key_version: "encryptionkey@0"
I20260925 14:48:30.057749 6374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:30.068405 6374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:30.070477 6374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:30.070662 6374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:30.070909 6374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:30.071249 6374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:30.071307 6374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:30.071352 6374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:30.071379 6374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:30.077852 6374 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36895
I20260925 14:48:30.077937 6504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36895 every 8 connection(s)
I20260925 14:48:30.078328 6374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:48:30.079278 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6374
Sep 25 14:48:30 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347710, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:48:30.087271 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.080175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42048 (local address 127.21.205.254:42575)
0925 14:48:30.080546 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.080548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.080553 (+ 5us) server_negotiation.cc:407] Connection header received
0925 14:48:30.080576 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.080577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.080587 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.080654 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:30.081278 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.081810 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.082583 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.082808 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.082886 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.084259 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:30.084260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:30.084261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:30.084268 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:30.085889 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.086354 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.086355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.086356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.086422 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.086659 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.086659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.086660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.086781 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:30.086964 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.087049 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.087164 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":88,"threads_started":1}
I20260925 14:48:30.089164 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.079481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:59277 (local address 127.21.205.254:42575)
0925 14:48:30.079533 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.079535 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.079657 (+ 122us) server_negotiation.cc:407] Connection header received
0925 14:48:30.080624 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.080626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.080637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.080683 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:30.081669 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.082171 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.083215 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.083470 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.083543 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.086027 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:30.086029 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:30.086029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:30.086037 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:30.087747 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.088263 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.088264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.088264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.088317 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.088525 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.088526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.088526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.088620 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:30.088664 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.089025 (+ 361us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.089097 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:48:30.089342 6507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.079186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:59277)
0925 14:48:30.079438 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.079528 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:30.080512 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:30.080766 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:30.080770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:30.081360 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:30.081569 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.081574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.082280 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.082281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.083127 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.083132 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.083559 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.083560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.083676 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.084233 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:30.084241 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:30.085916 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:30.087870 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:30.087879 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:30.087881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:30.088200 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:30.088380 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:30.088381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:30.088381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:30.088485 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:30.088672 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:30.088676 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:30.088889 (+ 213us) client_negotiation.cc:769] Sending connection context
0925 14:48:30.088981 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 14:48:30.089114 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260925 14:48:30.089826 6505 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42575
I20260925 14:48:30.089954 6505 heartbeater.cc:464] Registering TS with master...
I20260925 14:48:30.090152 6505 heartbeater.cc:511] Master 127.21.205.254:42575 requested a full tablet report, sending...
I20260925 14:48:30.090556 6039 ts_manager.cc:194] Registered new tserver with Master: 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
I20260925 14:48:30.090941 6039 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:59277
I20260925 14:48:30.098965 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:48:30.101794 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.099358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42060 (local address 127.21.205.254:42575)
0925 14:48:30.099422 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.099424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.099564 (+ 140us) server_negotiation.cc:407] Connection header received
0925 14:48:30.099627 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.099628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.099668 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.099724 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:48:30.099932 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.100419 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.101171 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.101358 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.101410 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.101550 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:48:30.101605 (+ 55us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 14:48:30.101728 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42060: token expired
Metrics: {"server-negotiator.queue_time_us":37}
W20260925 14:48:30.101888 6234 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42060: token expired
W20260925 14:48:30.102008 5964 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.205.254:42575)
I20260925 14:48:30.108191 6234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.102296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42072 (local address 127.21.205.254:42575)
0925 14:48:30.102367 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.102369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.102377 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:48:30.102420 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.102421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.102431 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.102479 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:30.102985 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.103459 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.104261 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.104463 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.104547 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.105735 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:30.105737 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:30.105737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:30.105744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:30.106959 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.107360 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.107361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.107362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.107423 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:30.107625 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:30.107626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:30.107626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:30.107750 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:30.107902 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.108001 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.108081 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 14:48:30.109411 6039 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:48:30.111024 6039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:48:30.113364 6164 tablet_service.cc:1467] Processing CreateTablet for tablet e1db874a929f40e78b2559a509a3bc61 (DEFAULT_TABLE table=test-table [id=c574d2390fee4ba2a330bf592ba814ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:30.113571 6164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1db874a929f40e78b2559a509a3bc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:30.115139 6121 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap starting.
I20260925 14:48:30.115504 6121 tablet_bootstrap.cc:654] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:30.117265 6511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.112926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36895 (local address 127.0.0.1:49418)
0925 14:48:30.113271 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.113280 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:30.113371 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:30.113591 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:30.113592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:30.113595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:30.113699 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.113706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.114852 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.114853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.116010 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.116012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.116978 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.116979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.117036 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.117056 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 14:48:30.117128 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 14:48:30.117176 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":95,"threads_started":1}
I20260925 14:48:30.117280 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.113022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.21.205.195:36895)
0925 14:48:30.113243 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.113250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.113360 (+ 110us) server_negotiation.cc:407] Connection header received
0925 14:48:30.113454 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.113457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.113476 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.113583 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:30.113784 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.114746 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.116096 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.116896 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.116980 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.117040 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.117150 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.117197 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":118,"threads_started":1}
I20260925 14:48:30.117411 6121 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: No bootstrap required, opened a new log
I20260925 14:48:30.117465 6121 ts_tablet_manager.cc:1397] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:48:30.117631 6121 raft_consensus.cc:348] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.117700 6121 raft_consensus.cc:374] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:30.117749 6121 raft_consensus.cc:729] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1dac631df44b086bce2f3e254bd3a, State: Initialized, Role: FOLLOWER
I20260925 14:48:30.117807 6121 consensus_queue.cc:260] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.117913 6121 ts_tablet_manager.cc:1428] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:30.118044 6438 tablet_service.cc:1467] Processing CreateTablet for tablet e1db874a929f40e78b2559a509a3bc61 (DEFAULT_TABLE table=test-table [id=c574d2390fee4ba2a330bf592ba814ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:30.118434 6438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1db874a929f40e78b2559a509a3bc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:30.119259 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.112883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38311 (local address 127.0.0.1:41932)
0925 14:48:30.112936 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.112947 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:30.112986 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:30.115447 (+ 2461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:30.115448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:30.115450 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:30.115493 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.115494 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.116745 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.116746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.117827 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.117830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.119043 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.119044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.119117 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.119121 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:48:30.119165 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 14:48:30.119206 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 14:48:30.119310 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.113074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.21.205.194:38311)
0925 14:48:30.115248 (+ 2174us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.115253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.115266 (+ 13us) server_negotiation.cc:407] Connection header received
0925 14:48:30.115323 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.115325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.115342 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.115437 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:30.115553 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.116669 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.118063 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.118926 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.119045 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.119107 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.119179 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.119223 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2124,"thread_start_us":94,"threads_started":1}
I20260925 14:48:30.120153 6305 tablet_service.cc:1467] Processing CreateTablet for tablet e1db874a929f40e78b2559a509a3bc61 (DEFAULT_TABLE table=test-table [id=c574d2390fee4ba2a330bf592ba814ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:48:30.120535 6305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1db874a929f40e78b2559a509a3bc61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:48:30.122188 6515 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Bootstrap starting.
I20260925 14:48:30.122794 6515 tablet_bootstrap.cc:654] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:30.123317 6515 log.cc:824] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:30.123929 6517 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Bootstrap starting.
I20260925 14:48:30.124195 6515 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: No bootstrap required, opened a new log
I20260925 14:48:30.124260 6515 ts_tablet_manager.cc:1397] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:48:30.124549 6517 tablet_bootstrap.cc:654] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Neither blocks nor log segments found. Creating new log.
I20260925 14:48:30.125157 6517 log.cc:824] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:30.125931 6517 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: No bootstrap required, opened a new log
I20260925 14:48:30.126014 6517 ts_tablet_manager.cc:1397] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:48:30.125960 6515 raft_consensus.cc:348] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.126160 6515 raft_consensus.cc:374] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:30.126206 6515 raft_consensus.cc:729] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e2c494548a49fdbb0e1ec7a1b06414, State: Initialized, Role: FOLLOWER
I20260925 14:48:30.126344 6515 consensus_queue.cc:260] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.126619 6515 ts_tablet_manager.cc:1428] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 14:48:30.126649 6505 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
I20260925 14:48:30.128309 6517 raft_consensus.cc:348] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.128477 6517 raft_consensus.cc:374] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:48:30.128526 6517 raft_consensus.cc:729] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870a3371a660461da0cc4ffd983cbfe0, State: Initialized, Role: FOLLOWER
I20260925 14:48:30.128647 6517 consensus_queue.cc:260] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.128963 6517 ts_tablet_manager.cc:1428] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:48:30.129019 6371 heartbeater.cc:503] Master 127.21.205.254:42575 was elected leader, sending a full tablet report...
I20260925 14:48:30.176405 6237 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:30.176561 6237 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.176959 6237 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
I20260925 14:48:30.181126 6513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.177295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:50739 (local address 127.21.205.194:38311)
0925 14:48:30.177347 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.177349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.177364 (+ 15us) server_negotiation.cc:407] Connection header received
0925 14:48:30.177400 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.177401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.177412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.177473 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:30.177690 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.178491 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180079 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.180689 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180775 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.180786 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.181008 (+ 222us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.181050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 14:48:30.181126 6512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.177258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:48151 (local address 127.21.205.195:36895)
0925 14:48:30.177313 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.177315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.177521 (+ 206us) server_negotiation.cc:407] Connection header received
0925 14:48:30.177554 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.177555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.177565 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.177617 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:48:30.177800 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.178490 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180080 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.180689 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180788 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.180796 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.181008 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.181050 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:48:30.181138 6233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.177165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38311 (local address 127.21.205.193:50739)
0925 14:48:30.177230 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.177245 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:30.177323 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:30.177494 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:30.177496 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:30.177498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:30.177588 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.177590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.178592 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.178593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.179971 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.179975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.180785 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.180890 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.180924 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 14:48:30.180988 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 14:48:30.181039 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":66}
I20260925 14:48:30.181144 6521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.177164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36895 (local address 127.21.205.193:48151)
0925 14:48:30.177417 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.177426 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:30.177504 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:30.177643 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:30.177644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:30.177646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:48:30.177706 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.177707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.178592 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.178593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.179971 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:30.179975 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.180796 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.180797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:30.180890 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.180924 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 14:48:30.180988 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 14:48:30.181039 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":96,"thread_start_us":106,"threads_started":1}
I20260925 14:48:30.181521 6458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1db874a929f40e78b2559a509a3bc61" candidate_uuid: "5ae1dac631df44b086bce2f3e254bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" is_pre_election: true
I20260925 14:48:30.181697 6458 raft_consensus.cc:2454] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1dac631df44b086bce2f3e254bd3a in term 0.
I20260925 14:48:30.181734 6325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1db874a929f40e78b2559a509a3bc61" candidate_uuid: "5ae1dac631df44b086bce2f3e254bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870a3371a660461da0cc4ffd983cbfe0" is_pre_election: true
I20260925 14:48:30.181912 6325 raft_consensus.cc:2454] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ae1dac631df44b086bce2f3e254bd3a in term 0.
I20260925 14:48:30.181953 6117 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a, 93e2c494548a49fdbb0e1ec7a1b06414; no voters:
I20260925 14:48:30.182089 6237 raft_consensus.cc:2781] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:48:30.182156 6237 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:48:30.182183 6237 raft_consensus.cc:3037] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:30.182896 6237 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.183054 6237 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 1 election: Requested vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
I20260925 14:48:30.183192 6325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1db874a929f40e78b2559a509a3bc61" candidate_uuid: "5ae1dac631df44b086bce2f3e254bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "870a3371a660461da0cc4ffd983cbfe0"
I20260925 14:48:30.183251 6458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1db874a929f40e78b2559a509a3bc61" candidate_uuid: "5ae1dac631df44b086bce2f3e254bd3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
I20260925 14:48:30.183280 6325 raft_consensus.cc:3037] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:30.183341 6458 raft_consensus.cc:3037] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:48:30.184119 6325 raft_consensus.cc:2454] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1dac631df44b086bce2f3e254bd3a in term 1.
I20260925 14:48:30.184125 6458 raft_consensus.cc:2454] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ae1dac631df44b086bce2f3e254bd3a in term 1.
I20260925 14:48:30.184293 6117 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a, 93e2c494548a49fdbb0e1ec7a1b06414; no voters:
I20260925 14:48:30.184407 6237 raft_consensus.cc:2781] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:48:30.184580 6237 raft_consensus.cc:686] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 LEADER]: Becoming Leader. State: Replica: 5ae1dac631df44b086bce2f3e254bd3a, State: Running, Role: LEADER
I20260925 14:48:30.184640 6237 consensus_queue.cc:237] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:30.185499 6038 catalog_manager.cc:5495] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193). New cstate: current_term: 1 leader_uuid: "5ae1dac631df44b086bce2f3e254bd3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } health_report { overall_health: HEALTHY } } }
I20260925 14:48:30.191424 6239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.188870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.21.205.193:37473)
0925 14:48:30.188986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:30.188988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:30.188997 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:48:30.189063 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:30.189064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:30.189080 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:30.189159 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:48:30.189306 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.189826 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.190592 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:30.190818 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:30.190896 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:30.191070 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:48:30.191183 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:48:30.191271 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:30.191305 (+ 34us) server_negotiation.cc:299] Negotiation successful
0925 14:48:30.191328 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260925 14:48:30.193011 6325 raft_consensus.cc:1261] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Refusing update from remote peer 5ae1dac631df44b086bce2f3e254bd3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:48:30.193011 6458 raft_consensus.cc:1261] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Refusing update from remote peer 5ae1dac631df44b086bce2f3e254bd3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:48:30.193413 6522 consensus_queue.cc:1035] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:48:30.193557 6237 consensus_queue.cc:1035] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:48:30.196987 6527 mvcc.cc:205] Tried to move back new op lower bound from 7333264220947984384 to 7333264220916965376. Current Snapshot: MvccSnapshot[applied={T|T < 7333264220947984384}]
I20260925 14:48:30.197067 6526 mvcc.cc:205] Tried to move back new op lower bound from 7333264220947984384 to 7333264220916965376. Current Snapshot: MvccSnapshot[applied={T|T < 7333264220947984384}]
I20260925 14:48:30.198379 6038 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:48:30.198486 6038 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:48:30.199311 6038 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Sending DeleteTablet for 3 replicas of tablet e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:30.199654 6164 tablet_service.cc:1514] Processing DeleteTablet for tablet e1db874a929f40e78b2559a509a3bc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:30 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:35120
I20260925 14:48:30.199668 6438 tablet_service.cc:1514] Processing DeleteTablet for tablet e1db874a929f40e78b2559a509a3bc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:30 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49418
I20260925 14:48:30.199694 6305 tablet_service.cc:1514] Processing DeleteTablet for tablet e1db874a929f40e78b2559a509a3bc61 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:48:30 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:41932
I20260925 14:48:30.199757 6240 tablet_replica.cc:330] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: stopping tablet replica
I20260925 14:48:30.199848 6240 raft_consensus.cc:2229] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:48:30.199941 6240 raft_consensus.cc:2258] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:30.199975 6536 tablet_replica.cc:330] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: stopping tablet replica
I20260925 14:48:30.200075 6536 raft_consensus.cc:2229] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:48:30.200296 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6078
I20260925 14:48:30.200397 6240 ts_tablet_manager.cc:1905] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:30.201486 6536 raft_consensus.cc:2258] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:30.201982 6536 ts_tablet_manager.cc:1905] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:30.203974 6536 ts_tablet_manager.cc:1918] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:48:30.203992 6537 tablet_replica.cc:330] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: stopping tablet replica
I20260925 14:48:30.204067 6536 log.cc:1196] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:30.204151 6537 raft_consensus.cc:2229] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:48:30.204319 6537 raft_consensus.cc:2258] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:48:30.204389 6536 ts_tablet_manager.cc:1939] T e1db874a929f40e78b2559a509a3bc61 P 93e2c494548a49fdbb0e1ec7a1b06414: Deleting consensus metadata
I20260925 14:48:30.204797 6537 ts_tablet_manager.cc:1905] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:48:30.204867 6025 catalog_manager.cc:4841] TS 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): tablet e1db874a929f40e78b2559a509a3bc61 (table test-table [id=c574d2390fee4ba2a330bf592ba814ac]) successfully deleted
I20260925 14:48:30.206334 6537 ts_tablet_manager.cc:1918] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:48:30.206434 6537 log.cc:1196] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:30.206701 6537 ts_tablet_manager.cc:1939] T e1db874a929f40e78b2559a509a3bc61 P 870a3371a660461da0cc4ffd983cbfe0: Deleting consensus metadata
I20260925 14:48:30.207208 6024 catalog_manager.cc:4841] TS 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): tablet e1db874a929f40e78b2559a509a3bc61 (table test-table [id=c574d2390fee4ba2a330bf592ba814ac]) successfully deleted
W20260925 14:48:30.207609 6025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:37473 (error 108)
I20260925 14:48:30.207850 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6236
I20260925 14:48:30.208927 6238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:30.208759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37473 (local address 127.0.0.1:35134)
0925 14:48:30.208821 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:30.208864 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 14:48:30.209055 6025 catalog_manager.cc:4584] TS 5ae1dac631df44b086bce2f3e254bd3a (127.21.205.193:37473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1db874a929f40e78b2559a509a3bc61: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37473: connect: Connection refused (error 111)
I20260925 14:48:30.215130 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6374
I20260925 14:48:30.220579 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6005
I20260925 14:48:50.226553 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--webserver_interface=127.21.205.254
--webserver_port=43753
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:50.308094 6539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:50.308314 6539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:50.308338 6539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:50.309563 6539 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 14:48:50.309610 6539 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 14:48:50.309624 6539 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 14:48:50.309636 6539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:48:50.309690 6539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:50.309716 6539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:50.309743 6539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:48:50.309763 6539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:48:50.311372 6539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=43753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:50.311648 6539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:50.311942 6539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:50.317442 6547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.317579 6539 server_base.cc:1034] running on GCE node
W20260925 14:48:50.317521 6545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:50.317442 6544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.318014 6539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:50.318334 6539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:50.319504 6539 hybrid_clock.cc:648] HybridClock initialized: now 1790347730319497 us; error 39 us; skew 500 ppm
Sep 25 14:48:50 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:50.325960 6539 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:48:50.326516 6539 webserver.cc:460] Webserver started at http://127.21.205.254:43753/ using document root <none> and password file <none>
I20260925 14:48:50.326781 6539 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:50.326853 6539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:50.328367 6539 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:48:50.329053 6554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.329227 6539 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:48:50.329340 6539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "869dcaae7d944adeabf285781dcf9246"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "206401f7a343d8bfddd28d33706fd73f"
server_key_iv: "81216471cc2de911b23d1eb6cc4b871e"
server_key_version: "encryptionkey@0"
I20260925 14:48:50.329721 6539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:50.345063 6539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:50.346993 6539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:50.347177 6539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:50.351413 6539 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42575
I20260925 14:48:50.351447 6606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42575 every 8 connection(s)
I20260925 14:48:50.351936 6539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:48:50.353444 6607 sys_catalog.cc:263] Verifying existing consensus state
I20260925 14:48:50.354045 6607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap starting.
I20260925 14:48:50.362098 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6539
I20260925 14:48:50.362509 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:37473
--local_ip_for_outbound_sockets=127.21.205.193
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=43675
--webserver_interface=127.21.205.193
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:48:50.363096 6607 log.cc:824] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:50.365583 6607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:48:50.365964 6607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Bootstrap complete.
I20260925 14:48:50.368461 6607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:50.368705 6607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Initialized, Role: FOLLOWER
I20260925 14:48:50.368888 6607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:50.368978 6607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:48:50.369035 6607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:48:50.369103 6607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 2 FOLLOWER]: Advancing to term 3
I20260925 14:48:50.370183 6607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:50.370322 6607 leader_election.cc:302] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 869dcaae7d944adeabf285781dcf9246; no voters:
I20260925 14:48:50.370529 6607 leader_election.cc:288] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 14:48:50.370568 6611 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 14:48:50.370692 6611 raft_consensus.cc:686] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [term 3 LEADER]: Becoming Leader. State: Replica: 869dcaae7d944adeabf285781dcf9246, State: Running, Role: LEADER
I20260925 14:48:50.370818 6607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:48:50.370850 6611 consensus_queue.cc:237] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } }
I20260925 14:48:50.371240 6612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:50.371254 6613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 869dcaae7d944adeabf285781dcf9246. Latest consensus state: current_term: 3 leader_uuid: "869dcaae7d944adeabf285781dcf9246" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "869dcaae7d944adeabf285781dcf9246" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42575 } } }
I20260925 14:48:50.371366 6612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:50.371378 6613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246 [sys.catalog]: This master's current role is: LEADER
I20260925 14:48:50.371598 6619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:48:50.372627 6619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:48:50.372804 6619 catalog_manager.cc:1259] Loaded cluster ID: c351fbd7f8d34918ac1dd1d2981920b1
I20260925 14:48:50.373039 6619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:48:50.374266 6619 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:48:50.374521 6619 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Loaded TSK: 0
I20260925 14:48:50.374621 6619 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 869dcaae7d944adeabf285781dcf9246: Loaded TSK: 1
F20260925 14:48:50.374953 6619 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f11972569ad google::LogMessage::Fail() at ??:0
@ 0x7f119725988a google::LogMessage::SendToLog() at ??:0
@ 0x7f11972564de google::LogMessage::Flush() at ??:0
@ 0x7f1197257a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f11999c3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f11999c0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f119f1b01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f119f1b048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f119f1b0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f119f176c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f119f180ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f11991240d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1199119954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1199f7d6db start_thread at ??:0
@ 0x7f1195d5461f clone at ??:0
W20260925 14:48:50.458988 6610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:50.459254 6610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:50.459296 6610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:50.460647 6610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:50.460719 6610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:50.460769 6610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:48:50.462379 6610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:37473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=43675
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:50.462682 6610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:50.462932 6610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:50.469195 6648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:50.469205 6649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.469372 6610 server_base.cc:1034] running on GCE node
W20260925 14:48:50.469524 6651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.469761 6610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:50.470050 6610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:50.471232 6610 hybrid_clock.cc:648] HybridClock initialized: now 1790347730471216 us; error 34 us; skew 500 ppm
Sep 25 14:48:50 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:50.473942 6610 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:48:50.474474 6610 webserver.cc:460] Webserver started at http://127.21.205.193:43675/ using document root <none> and password file <none>
I20260925 14:48:50.474725 6610 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:50.474794 6610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:50.476398 6610 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260925 14:48:50.477283 6658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.477494 6610 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:50.477598 6610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "5ae1dac631df44b086bce2f3e254bd3a"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "05d4768cb52fb2651405241150130633"
server_key_iv: "6605a2c88da68ffe5da02c9895daa36d"
server_key_version: "encryptionkey@0"
I20260925 14:48:50.477988 6610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:50.492981 6610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:50.495532 6610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:50.495712 6610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:50.495990 6610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:50.496469 6665 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260925 14:48:50.498556 6610 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 14:48:50.498646 6610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260925 14:48:50.498718 6610 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 14:48:50.499392 6610 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 14:48:50.499446 6610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260925 14:48:50.499529 6665 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap starting.
I20260925 14:48:50.501480 6665 log.cc:824] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Log is configured to *not* fsync() on all Append() calls
I20260925 14:48:50.504719 6665 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 14:48:50.505055 6665 tablet_bootstrap.cc:492] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Bootstrap complete.
I20260925 14:48:50.505230 6665 ts_tablet_manager.cc:1397] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260925 14:48:50.506639 6610 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37473
I20260925 14:48:50.506673 6774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37473 every 8 connection(s)
I20260925 14:48:50.507035 6610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:48:50.507170 6665 raft_consensus.cc:348] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:50.507597 6665 raft_consensus.cc:729] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ae1dac631df44b086bce2f3e254bd3a, State: Initialized, Role: FOLLOWER
I20260925 14:48:50.507808 6665 consensus_queue.cc:260] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:50.508188 6665 ts_tablet_manager.cc:1428] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:48:50.508184 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6610
I20260925 14:48:50.508507 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:38311
--local_ip_for_outbound_sockets=127.21.205.194
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=36033
--webserver_interface=127.21.205.194
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:50.604146 6777 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:48:50.507757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:55379)
0925 14:48:50.507982 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.508032 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:50.508200 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:50.603938 (+ 95738us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
W20260925 14:48:50.604413 6775 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:42575 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:42575: Client connection negotiation failed: client connection to 127.21.205.254:42575: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 14:48:50.604750 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.604628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:51811)
0925 14:48:50.604683 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.604726 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 14:48:50.605062 6777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.604964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:60365)
0925 14:48:50.605010 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.605040 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 14:48:50.605185 6775 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 14:48:50.617902 6779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:50.618090 6779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:50.618114 6779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:50.619221 6779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:50.619261 6779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:50.619315 6779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:48:50.620783 6779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:38311
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=36033
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:50.621019 6779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:50.621250 6779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:50.626092 6787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:50.626097 6785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.626246 6779 server_base.cc:1034] running on GCE node
W20260925 14:48:50.626101 6784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.626529 6779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:50.626780 6779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:50.627931 6779 hybrid_clock.cc:648] HybridClock initialized: now 1790347730627919 us; error 39 us; skew 500 ppm
Sep 25 14:48:50 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:50.630182 6779 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:48:50.630606 6779 webserver.cc:460] Webserver started at http://127.21.205.194:36033/ using document root <none> and password file <none>
I20260925 14:48:50.630838 6779 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:50.630903 6779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:50.632200 6779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:48:50.632757 6794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.632910 6779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:48:50.633006 6779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "870a3371a660461da0cc4ffd983cbfe0"
format_stamp: "Formatted at 2026-09-25 14:48:08 on dist-test-slave-d7lw"
server_key: "fd3678e66aa7ecb3497c6c22051faceb"
server_key_iv: "9d9e3317f9886a00e33cecf26fa88771"
server_key_version: "encryptionkey@0"
I20260925 14:48:50.633307 6779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:50.646924 6779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:50.648726 6779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:50.648878 6779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:50.649092 6779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:50.649384 6779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:50.649433 6779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.649473 6779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:50.649501 6779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.655301 6779 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38311
I20260925 14:48:50.655354 6911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38311 every 8 connection(s)
I20260925 14:48:50.655679 6779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:48:50.656657 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.656290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:57551)
0925 14:48:50.656433 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.656498 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
W20260925 14:48:50.657016 6912 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:42575 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:42575: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
I20260925 14:48:50.657281 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.657193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:43145)
0925 14:48:50.657237 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.657264 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 14:48:50.657546 6914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.657466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:54651)
0925 14:48:50.657504 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.657525 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260925 14:48:50.657644 6912 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 14:48:50.664259 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6779
I20260925 14:48:50.664631 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:36895
--local_ip_for_outbound_sockets=127.21.205.195
--tserver_master_addrs=127.21.205.254:42575
--webserver_port=33787
--webserver_interface=127.21.205.195
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:48:50.744563 6915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:48:50.744762 6915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:48:50.744797 6915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:48:50.745858 6915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:48:50.745916 6915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:48:50.745959 6915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:48:50.747323 6915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:36895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=33787
--tserver_master_addrs=127.21.205.254:42575
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:48:50.747560 6915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:48:50.747766 6915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:48:50.752556 6923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:50.752578 6920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:48:50.752568 6921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:48:50.752708 6915 server_base.cc:1034] running on GCE node
I20260925 14:48:50.753005 6915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:48:50.753242 6915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:48:50.754396 6915 hybrid_clock.cc:648] HybridClock initialized: now 1790347730754387 us; error 26 us; skew 500 ppm
Sep 25 14:48:50 dist-test-slave-d7lw krb5kdc[5448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:48:50.756816 6915 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:48:50.757157 6915 webserver.cc:460] Webserver started at http://127.21.205.195:33787/ using document root <none> and password file <none>
I20260925 14:48:50.757349 6915 fs_manager.cc:359] Metadata directory not provided
I20260925 14:48:50.757388 6915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:48:50.758574 6915 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:48:50.759145 6930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.759336 6915 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:48:50.759409 6915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "93e2c494548a49fdbb0e1ec7a1b06414"
format_stamp: "Formatted at 2026-09-25 14:48:09 on dist-test-slave-d7lw"
server_key: "95e76fbee349595988da60ce8593f1be"
server_key_iv: "073f774cc4e80ad4939d59f762c3ef4c"
server_key_version: "encryptionkey@0"
I20260925 14:48:50.759693 6915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:48:50.768052 6915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:48:50.769654 6915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:48:50.769761 6915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:48:50.769928 6915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:48:50.770138 6915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:48:50.770165 6915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.770184 6915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:48:50.770196 6915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:48:50.775494 6915 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:36895
I20260925 14:48:50.775578 7045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:36895 every 8 connection(s)
I20260925 14:48:50.775918 6915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:48:50.776903 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.776513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:33239)
0925 14:48:50.776675 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.776761 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
W20260925 14:48:50.777351 7046 heartbeater.cc:644] Failed to heartbeat to 127.21.205.254:42575 (0 consecutive failures): Network error: Failed to ping master at 127.21.205.254:42575: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
I20260925 14:48:50.777575 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.777512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:41855)
0925 14:48:50.777537 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.777561 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
I20260925 14:48:50.777781 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:50.777736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:60933)
0925 14:48:50.777754 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:50.777772 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260925 14:48:50.777863 7046 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 14:48:50.778995 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 6915
I20260925 14:48:51.605742 7050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:51.605531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:50459)
0925 14:48:51.605643 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:51.605679 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260925 14:48:51.658182 7051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:51.657927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:43847)
0925 14:48:51.658104 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:51.658137 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":52,"threads_started":1}
I20260925 14:48:51.778314 7052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:51.778131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:37511)
0925 14:48:51.778232 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:51.778269 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":50,"threads_started":1}
I20260925 14:48:52.359987 7053 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:52.360106 7053 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:52.360425 7053 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
Sep 25 14:48:52 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.195@KRBTEST.COM
Sep 25 14:48:52 dist-test-slave-d7lw krb5kdc[5448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347730, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.194@KRBTEST.COM
I20260925 14:48:52.371490 7057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.360707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:58795 (local address 127.21.205.195:36895)
0925 14:48:52.360865 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:52.360878 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:52.360903 (+ 25us) server_negotiation.cc:407] Connection header received
0925 14:48:52.361339 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:52.361341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:52.361383 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:52.361568 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:52.362507 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.363437 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.364461 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.364919 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.365029 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:52.367775 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:52.367796 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:52.367796 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:52.367810 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:52.369733 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370258 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:52.370262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:52.370262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:52.370370 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370625 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:52.370626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:52.370627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:52.370865 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:52.370957 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:52.371232 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 14:48:52.371369 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":47,"threads_started":1}
I20260925 14:48:52.371495 7056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.360699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:49963 (local address 127.21.205.194:38311)
0925 14:48:52.360888 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 14:48:52.360903 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 14:48:52.360921 (+ 18us) server_negotiation.cc:407] Connection header received
0925 14:48:52.361339 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:48:52.361341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:48:52.361382 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:48:52.361572 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:48:52.362329 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.363383 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.364454 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.364914 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.365019 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:52.368223 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:48:52.368226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:48:52.368227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:48:52.368237 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:48:52.369874 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370287 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:52.370289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:52.370290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:52.370370 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370620 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:48:52.370621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:48:52.370622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:48:52.370860 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:48:52.370945 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 14:48:52.371237 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 14:48:52.371362 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":41,"threads_started":1}
I20260925 14:48:52.371495 7055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.360605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38311 (local address 127.21.205.193:49963)
0925 14:48:52.360765 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:52.360790 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:52.360852 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:52.361569 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:52.361572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:52.361988 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:52.362204 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:52.362210 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.363478 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.363479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:52.364368 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:52.364371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.365106 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.365107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:52.365210 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:52.365848 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:52.365852 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:52.368114 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:52.369952 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:52.369952 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:52.369953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:52.370177 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:52.370437 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370438 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:52.370438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:52.370553 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:52.370928 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:52.370934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:52.371071 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 14:48:52.371172 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 14:48:52.371370 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":23,"thread_start_us":72,"threads_started":1}
I20260925 14:48:52.371590 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.360604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:36895 (local address 127.21.205.193:58795)
0925 14:48:52.360737 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:52.360748 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:48:52.360821 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:48:52.361569 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:48:52.361572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:48:52.361994 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:48:52.362403 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:52.362406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.363528 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.363529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:52.364370 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:48:52.364373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:48:52.365027 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:48:52.365028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:48:52.365210 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:48:52.365818 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:48:52.365829 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:48:52.367657 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:48:52.369824 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:52.369832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:52.369834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:52.370168 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:52.370439 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:48:52.370440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:48:52.370440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:48:52.370553 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:48:52.370953 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:48:52.370954 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:48:52.371071 (+ 117us) client_negotiation.cc:769] Sending connection context
0925 14:48:52.371173 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 14:48:52.371448 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":186,"thread_start_us":64,"threads_started":1}
W20260925 14:48:52.372557 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:48:52.372671 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:52.372710 6661 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:48:52.372812 7053 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:48:52.606573 7054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.606396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:50695)
0925 14:48:52.606489 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:52.606531 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260925 14:48:52.658958 7058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.658695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:38205)
0925 14:48:52.658855 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:52.658913 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
I20260925 14:48:52.778925 7059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:52.778703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:47667)
0925 14:48:52.778841 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:52.778885 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260925 14:48:53.607303 7060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:53.607071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:45213)
0925 14:48:53.607213 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:53.607247 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260925 14:48:53.659605 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:53.659408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:43955)
0925 14:48:53.659521 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:53.659565 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":53,"threads_started":1}
I20260925 14:48:53.779443 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:53.779278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:35257)
0925 14:48:53.779384 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:53.779410 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":51,"threads_started":1}
I20260925 14:48:54.347846 7063 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:54.347934 7063 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:54.348073 7063 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
W20260925 14:48:54.348469 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:48:54.348541 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:54.348584 6661 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:48:54.348664 7063 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:48:54.607929 7064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:54.607713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:55977)
0925 14:48:54.607851 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:54.607883 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":106,"threads_started":1}
I20260925 14:48:54.660208 7065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:54.659989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:45599)
0925 14:48:54.660111 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:54.660144 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260925 14:48:54.780161 7066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:54.779945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:44937)
0925 14:48:54.780087 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:54.780120 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260925 14:48:55.608680 7067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:55.608451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:49299)
0925 14:48:55.608582 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:55.608627 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":65,"threads_started":1}
I20260925 14:48:55.660882 7068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:55.660674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:51397)
0925 14:48:55.660806 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:55.660842 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260925 14:48:55.780725 7069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:55.780548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:33613)
0925 14:48:55.780655 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:55.780689 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":58,"threads_started":1}
I20260925 14:48:56.609449 7070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:56.609226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:54573)
0925 14:48:56.609356 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:56.609397 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260925 14:48:56.661548 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:56.661329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:44421)
0925 14:48:56.661450 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:56.661500 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260925 14:48:56.781333 7072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:56.781118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:54043)
0925 14:48:56.781227 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:56.781259 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260925 14:48:57.610162 7073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:57.609966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:37219)
0925 14:48:57.610089 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:57.610117 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260925 14:48:57.632565 7074 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:57.632643 7074 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:57.632792 7074 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
W20260925 14:48:57.633165 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:48:57.633249 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:57.633342 6660 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:48:57.633430 7074 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:48:57.662169 7075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:57.661984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:46031)
0925 14:48:57.662102 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:57.662135 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260925 14:48:57.781914 7076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:57.781734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:34265)
0925 14:48:57.781846 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:57.781876 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260925 14:48:58.610898 7077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:58.610681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:59611)
0925 14:48:58.610815 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:58.610851 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":99,"threads_started":1}
I20260925 14:48:58.662732 7078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:58.662552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:33355)
0925 14:48:58.662662 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:58.662695 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260925 14:48:58.782470 7079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:58.782304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:44909)
0925 14:48:58.782406 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:58.782435 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
I20260925 14:48:59.611755 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:59.611507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:34671)
0925 14:48:59.611677 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:59.611709 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":108,"threads_started":1}
I20260925 14:48:59.663388 7081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:59.663163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:49781)
0925 14:48:59.663288 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:59.663338 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":62,"threads_started":1}
I20260925 14:48:59.783099 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:48:59.782900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:33169)
0925 14:48:59.783024 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 14:48:59.783059 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260925 14:48:59.940864 7083 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:48:59.940989 7083 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:48:59.941167 7083 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
W20260925 14:48:59.941619 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:48:59.941720 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:48:59.941787 6661 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:48:59.941884 7083 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:49:00.612610 7084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:00.612346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:58219)
0925 14:49:00.612483 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:00.612525 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260925 14:49:00.664038 7085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:00.663838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:58697)
0925 14:49:00.663965 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:00.663998 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260925 14:49:00.783633 7086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:00.783445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:45301)
0925 14:49:00.783568 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:00.783597 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260925 14:49:01.540766 7087 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:01.540858 7087 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:49:01.541069 7087 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
W20260925 14:49:01.541457 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:49:01.541508 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:49:01.541553 6661 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:49:01.541648 7087 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:49:01.613296 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:01.613089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:44495)
0925 14:49:01.613205 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:01.613238 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260925 14:49:01.664669 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:01.664479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:46601)
0925 14:49:01.664591 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:01.664626 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":57,"threads_started":1}
I20260925 14:49:01.784250 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:01.784058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:43127)
0925 14:49:01.784177 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:01.784210 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260925 14:49:02.613965 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:02.613754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:59053)
0925 14:49:02.613882 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:02.613917 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260925 14:49:02.665306 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:02.665053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:60365)
0925 14:49:02.665163 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:02.665231 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":55,"threads_started":1}
I20260925 14:49:02.785073 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:02.784770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:46845)
0925 14:49:02.785001 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:02.785032 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":56,"threads_started":1}
I20260925 14:49:03.614590 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:03.614386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:36629)
0925 14:49:03.614493 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:03.614533 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
I20260925 14:49:03.665885 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:03.665690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:34733)
0925 14:49:03.665816 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:03.665850 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260925 14:49:03.785630 7096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:03.785443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:33417)
0925 14:49:03.785568 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:03.785596 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":52,"threads_started":1}
I20260925 14:49:04.615451 7097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:04.615231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:47971)
0925 14:49:04.615368 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:04.615403 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":73,"threads_started":1}
I20260925 14:49:04.666529 7098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:04.666320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:55519)
0925 14:49:04.666455 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:04.666487 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260925 14:49:04.783236 7099 raft_consensus.cc:482] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:04.783326 7099 raft_consensus.cc:504] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a3371a660461da0cc4ffd983cbfe0" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38311 } } peers { permanent_uuid: "93e2c494548a49fdbb0e1ec7a1b06414" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 36895 } } peers { permanent_uuid: "5ae1dac631df44b086bce2f3e254bd3a" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37473 } }
I20260925 14:49:04.783527 7099 leader_election.cc:288] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311), 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895)
W20260925 14:49:04.783910 6660 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 870a3371a660461da0cc4ffd983cbfe0 (127.21.205.194:38311): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
W20260925 14:49:04.783967 6661 leader_election.cc:341] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 93e2c494548a49fdbb0e1ec7a1b06414 (127.21.205.195:36895): Not found: Tablet not found: e1db874a929f40e78b2559a509a3bc61
I20260925 14:49:04.784018 6661 leader_election.cc:302] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ae1dac631df44b086bce2f3e254bd3a; no voters: 870a3371a660461da0cc4ffd983cbfe0, 93e2c494548a49fdbb0e1ec7a1b06414
I20260925 14:49:04.784086 7099 raft_consensus.cc:2726] T e1db874a929f40e78b2559a509a3bc61 P 5ae1dac631df44b086bce2f3e254bd3a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 14:49:04.786164 7100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:04.785981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:37443)
0925 14:49:04.786082 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:04.786112 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":63,"threads_started":1}
I20260925 14:49:05.616358 7101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:05.616137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.193:50435)
0925 14:49:05.616279 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:05.616312 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260925 14:49:05.667203 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:05.667002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.194:33393)
0925 14:49:05.667128 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:05.667162 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260925 14:49:05.783922 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6610
I20260925 14:49:05.786862 7103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:05.786658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42575 (local address 127.21.205.195:36033)
0925 14:49:05.786779 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:05.786820 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.254:42575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260925 14:49:05.789000 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6779
I20260925 14:49:05.793411 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 6915
2026-09-25T14:49:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58483 ms)
[----------] 30 tests from SecurityITest (259365 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:05 dist-test-slave-d7lw krb5kdc[7106](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:05 dist-test-slave-d7lw krb5kdc[7106](info): set up 2 sockets
Sep 25 14:49:05 dist-test-slave-d7lw krb5kdc[7106](info): commencing operation
krb5kdc: starting...
W20260925 14:49:06.814287 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:06 dist-test-slave-d7lw krb5kdc[7106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:06.950724 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:41113
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:41113
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:07.029397 7120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.029623 7120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.029660 7120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.030800 7120 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:07.030865 7120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.030890 7120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.030920 7120 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:07.030951 7120 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:07.032480 7120 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:41113
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:41113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.032747 7120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.032975 7120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.037638 7128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.037638 7125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.037693 7126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.037880 7120 server_base.cc:1034] running on GCE node
I20260925 14:49:07.038129 7120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.038431 7120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.039599 7120 hybrid_clock.cc:648] HybridClock initialized: now 1790347747039571 us; error 26 us; skew 500 ppm
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:07.041961 7120 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:07.042369 7120 webserver.cc:460] Webserver started at http://127.21.205.254:35881/ using document root <none> and password file <none>
I20260925 14:49:07.042610 7120 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.042655 7120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.042804 7120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.043887 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "49bfcaa49b6c44d8af031ab27ba7db4b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "cc17d59994c2200bd06b2445336fef14"
server_key_iv: "c6899abf0ffc2e2ff9536531f09f2a15"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.044247 7120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "49bfcaa49b6c44d8af031ab27ba7db4b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "cc17d59994c2200bd06b2445336fef14"
server_key_iv: "c6899abf0ffc2e2ff9536531f09f2a15"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.045512 7120 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:07.046201 7136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.046391 7120 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:07.046470 7120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "49bfcaa49b6c44d8af031ab27ba7db4b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "cc17d59994c2200bd06b2445336fef14"
server_key_iv: "c6899abf0ffc2e2ff9536531f09f2a15"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.046541 7120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:07.058347 7120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:07.060094 7120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:07.060240 7120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:07.063994 7120 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:41113
I20260925 14:49:07.064028 7188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:41113 every 8 connection(s)
I20260925 14:49:07.064354 7120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:07.064772 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7120
I20260925 14:49:07.064899 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:07.065111 22327 external_mini_cluster.cc:1428] Setting key e63dffb3bee80a21fa410e6f1945c53e
I20260925 14:49:07.065388 7189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:07.068099 7189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: Bootstrap starting.
I20260925 14:49:07.068665 7189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:07.068949 7189 log.cc:824] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:07.069588 7189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: No bootstrap required, opened a new log
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347746, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:07.071862 7189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } }
I20260925 14:49:07.071990 7189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:07.072013 7189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49bfcaa49b6c44d8af031ab27ba7db4b, State: Initialized, Role: FOLLOWER
I20260925 14:49:07.072151 7189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } }
I20260925 14:49:07.072230 7189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:07.072279 7189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:07.072381 7189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:07.073096 7189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } }
I20260925 14:49:07.073215 7189 leader_election.cc:302] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49bfcaa49b6c44d8af031ab27ba7db4b; no voters:
I20260925 14:49:07.073395 7189 leader_election.cc:288] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:07.073447 7194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:07.073585 7194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [term 1 LEADER]: Becoming Leader. State: Replica: 49bfcaa49b6c44d8af031ab27ba7db4b, State: Running, Role: LEADER
I20260925 14:49:07.073666 7189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:07.073769 7194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } }
I20260925 14:49:07.074147 7194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49bfcaa49b6c44d8af031ab27ba7db4b. Latest consensus state: current_term: 1 leader_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } } }
I20260925 14:49:07.074151 7196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49bfcaa49b6c44d8af031ab27ba7db4b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 41113 } } }
I20260925 14:49:07.074319 7196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:07.074568 7194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:07.074954 7203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:07.075829 7203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:07.076575 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.065612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50250 (local address 127.21.205.254:41113)
0925 14:49:07.065796 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.065807 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.065837 (+ 30us) server_negotiation.cc:407] Connection header received
0925 14:49:07.066456 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.066463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.066507 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.066684 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.067295 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.068302 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.069136 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.069539 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.069673 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.072254 (+ 2581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.072259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.072259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:07.072271 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.074253 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.074794 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.074797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.074798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.074905 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.075249 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.075249 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.075250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.075577 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.076171 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.076241 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.076373 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260925 14:49:07.077417 7203 catalog_manager.cc:1347] Generated new cluster ID: 7e38c3049aa6485985f9914bb6e1e2e7
I20260925 14:49:07.077456 7203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:07.083775 7203 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:07.084399 7203 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:07.087021 7203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: Generated new TSK 0
I20260925 14:49:07.087193 7203 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:49:07.112376 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:07.191246 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.191432 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.191455 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.192648 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.192689 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.192716 7215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:07.194120 7215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.194348 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.194591 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.199473 7223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.199419 7220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.199419 7221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.199615 7215 server_base.cc:1034] running on GCE node
I20260925 14:49:07.199884 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.200135 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.201282 7215 hybrid_clock.cc:648] HybridClock initialized: now 1790347747201250 us; error 54 us; skew 500 ppm
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:07.203590 7215 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:49:07.204056 7215 webserver.cc:460] Webserver started at http://127.21.205.193:33809/ using document root <none> and password file <none>
I20260925 14:49:07.204281 7215 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.204349 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.204483 7215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.205489 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "b5b66bb7d960431cbbb83e18db78ee2b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "bd03d3ad4955f14ebee089718cb96361"
server_key_iv: "3b17424d67e11431557b2628b07ad6f7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.205852 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "b5b66bb7d960431cbbb83e18db78ee2b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "bd03d3ad4955f14ebee089718cb96361"
server_key_iv: "3b17424d67e11431557b2628b07ad6f7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.207113 7215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:07.207832 7230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.208029 7215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:07.208124 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "b5b66bb7d960431cbbb83e18db78ee2b"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "bd03d3ad4955f14ebee089718cb96361"
server_key_iv: "3b17424d67e11431557b2628b07ad6f7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.208209 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:07.254154 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:07.255983 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:07.256152 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:07.256385 7215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:07.256714 7215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:07.256776 7215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.256817 7215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:07.256845 7215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.262360 7215 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:45269
I20260925 14:49:07.262410 7344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:45269 every 8 connection(s)
I20260925 14:49:07.262707 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:49:07.266976 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7215
I20260925 14:49:07.267102 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:07.267333 22327 external_mini_cluster.cc:1428] Setting key 9729f987637fdb6494caa35ba693494b
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:07.272245 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.263365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:47727 (local address 127.21.205.254:41113)
0925 14:49:07.263434 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.263436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.263626 (+ 190us) server_negotiation.cc:407] Connection header received
0925 14:49:07.264173 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.264174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.264183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.264241 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.265132 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.265630 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.266575 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.266787 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.266836 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.269336 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.269338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.269338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:07.269346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.270810 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.271282 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.271282 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.271283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.271329 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.271609 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.271609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.271610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.271700 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.271768 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.272117 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.272184 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:49:07.272519 7347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.263317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41113 (local address 127.21.205.193:47727)
0925 14:49:07.263492 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.263533 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.264086 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.264308 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.264311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.264853 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:07.265041 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.265046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.265712 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.265712 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.266504 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.266506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.266868 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.266869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.267018 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.267540 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:07.267564 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:07.269230 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:07.270889 (+ 1659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.270896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.270899 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.271205 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.271388 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.271389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.271389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.271535 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.271778 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:07.271802 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:07.271977 (+ 175us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.272098 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.272265 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260925 14:49:07.273144 7345 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41113
I20260925 14:49:07.273236 7345 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:07.273427 7345 heartbeater.cc:511] Master 127.21.205.254:41113 requested a full tablet report, sending...
I20260925 14:49:07.273888 7153 ts_manager.cc:194] Registered new tserver with Master: b5b66bb7d960431cbbb83e18db78ee2b (127.21.205.193:45269)
I20260925 14:49:07.274597 7153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:47727
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:49:07.292016 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:07.371927 7350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.372150 7350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.372190 7350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.373436 7350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.373495 7350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.373540 7350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:07.375151 7350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.375406 7350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.375690 7350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.380568 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.380618 7358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.380609 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.380928 7350 server_base.cc:1034] running on GCE node
I20260925 14:49:07.381147 7350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.381405 7350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.382601 7350 hybrid_clock.cc:648] HybridClock initialized: now 1790347747382571 us; error 24 us; skew 500 ppm
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:07.385022 7350 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:49:07.385476 7350 webserver.cc:460] Webserver started at http://127.21.205.194:34493/ using document root <none> and password file <none>
I20260925 14:49:07.385721 7350 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.385795 7350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.385941 7350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.387039 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "f8952875df5c444f8f0543b9c6a03ed2"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b7bb8d50758d60bdd3defd97b561f9e0"
server_key_iv: "7df016497b9bf4d357b259935514d100"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.387408 7350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "f8952875df5c444f8f0543b9c6a03ed2"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b7bb8d50758d60bdd3defd97b561f9e0"
server_key_iv: "7df016497b9bf4d357b259935514d100"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.388792 7350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:07.389533 7365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.389709 7350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:07.389818 7350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "f8952875df5c444f8f0543b9c6a03ed2"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b7bb8d50758d60bdd3defd97b561f9e0"
server_key_iv: "7df016497b9bf4d357b259935514d100"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.389894 7350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:07.410439 7350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:07.412390 7350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:07.412559 7350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:07.412801 7350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:07.413111 7350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:07.413165 7350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.413206 7350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:07.413233 7350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.419236 7350 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:44393
I20260925 14:49:07.419288 7480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:44393 every 8 connection(s)
I20260925 14:49:07.419603 7350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:07.426148 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7350
I20260925 14:49:07.426280 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:07.426518 22327 external_mini_cluster.cc:1428] Setting key 9d91a77a5fa74a97f9f4d7bd9f4bd3ca
I20260925 14:49:07.429237 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.420315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:50167 (local address 127.21.205.254:41113)
0925 14:49:07.420366 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.420367 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.420570 (+ 203us) server_negotiation.cc:407] Connection header received
0925 14:49:07.421069 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.421070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.421079 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.421147 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.422061 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.422562 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.423514 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.423704 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.423758 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.425887 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.425888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.425889 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:07.425895 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.427724 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.428142 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.428143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.428144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.428204 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.428432 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.428433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.428433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.428578 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.428653 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.429109 (+ 456us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.429174 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:07.429531 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.420294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41113 (local address 127.21.205.194:50167)
0925 14:49:07.420461 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.420501 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.420984 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.421224 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.421227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.421781 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:07.421978 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.421983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.422643 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.422644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.423440 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.423442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.423778 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.423778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.423896 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.424408 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:07.424424 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:07.425793 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:07.427850 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.427856 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.427858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.428070 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.428275 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.428275 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.428276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.428371 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.428762 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:07.428770 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:07.428953 (+ 183us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.429086 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.429270 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260925 14:49:07.430086 7481 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41113
I20260925 14:49:07.430200 7481 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:07.430408 7481 heartbeater.cc:511] Master 127.21.205.254:41113 requested a full tablet report, sending...
I20260925 14:49:07.430809 7153 ts_manager.cc:194] Registered new tserver with Master: f8952875df5c444f8f0543b9c6a03ed2 (127.21.205.194:44393)
I20260925 14:49:07.431221 7153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:50167
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:49:07.450905 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:07.531661 7486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.531874 7486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.531898 7486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.532974 7486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.533008 7486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.533034 7486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:07.534366 7486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:41113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.534534 7486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.534698 7486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.539520 7491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.539496 7492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.539654 7486 server_base.cc:1034] running on GCE node
W20260925 14:49:07.539500 7494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.539956 7486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.540196 7486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.541360 7486 hybrid_clock.cc:648] HybridClock initialized: now 1790347747541338 us; error 40 us; skew 500 ppm
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:07.543704 7486 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:49:07.544184 7486 webserver.cc:460] Webserver started at http://127.21.205.195:37263/ using document root <none> and password file <none>
I20260925 14:49:07.544409 7486 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.544476 7486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.544605 7486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.545603 7486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "f70a756e2f8240a4bd99136f9c89cc88"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "e69f966a085936b516f84d9721204c0e"
server_key_iv: "fb49fae6a69e34399fee6d998d4cdca5"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.545964 7486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "f70a756e2f8240a4bd99136f9c89cc88"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "e69f966a085936b516f84d9721204c0e"
server_key_iv: "fb49fae6a69e34399fee6d998d4cdca5"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.547272 7486 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:49:07.548300 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.548516 7486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:07.548620 7486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "f70a756e2f8240a4bd99136f9c89cc88"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "e69f966a085936b516f84d9721204c0e"
server_key_iv: "fb49fae6a69e34399fee6d998d4cdca5"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.548731 7486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:07.557621 7486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:07.559300 7486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:07.559442 7486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:07.559660 7486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:07.559978 7486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:07.560015 7486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.560067 7486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:07.560108 7486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.565560 7486 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:42249
I20260925 14:49:07.565630 7614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:42249 every 8 connection(s)
I20260925 14:49:07.565930 7486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:49:07 dist-test-slave-d7lw krb5kdc[7106](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347747, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:07.574678 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.566635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:49043 (local address 127.21.205.254:41113)
0925 14:49:07.566699 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.566701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.566872 (+ 171us) server_negotiation.cc:407] Connection header received
0925 14:49:07.567329 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.567330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.567340 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.567401 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.568407 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.568924 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.569941 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.570144 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.570202 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.572238 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.572239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.572240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:07.572246 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.573431 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.573822 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.573823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.573824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.573885 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.574136 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.574137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.574137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.574234 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.574286 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.574546 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.574614 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 14:49:07.574836 7617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.566601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:41113 (local address 127.21.205.195:49043)
0925 14:49:07.566755 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.566793 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.567238 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.567455 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.567457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.568134 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:07.568329 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.568333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.569002 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.569002 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.569863 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.569865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.570217 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.570218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.570318 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.570805 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:07.570813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:07.572153 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:07.573500 (+ 1347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.573506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.573508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.573704 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.573967 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:07.573968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:07.573968 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:07.574078 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:07.574290 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:07.574295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:07.574437 (+ 142us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.574518 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.574630 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260925 14:49:07.575392 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7486
I20260925 14:49:07.575527 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:07.575520 7615 heartbeater.cc:347] Connected to a master server at 127.21.205.254:41113
I20260925 14:49:07.575621 7615 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:07.575735 22327 external_mini_cluster.cc:1428] Setting key ccb5bc4022731c9f3cd267bd0b0a6624
I20260925 14:49:07.575835 7615 heartbeater.cc:511] Master 127.21.205.254:41113 requested a full tablet report, sending...
I20260925 14:49:07.576220 7152 ts_manager.cc:194] Registered new tserver with Master: f70a756e2f8240a4bd99136f9c89cc88 (127.21.205.195:42249)
I20260925 14:49:07.576586 7152 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:49043
I20260925 14:49:07.577241 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:07.584488 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.578524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.21.205.254:41113)
0925 14:49:07.578597 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.578599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.578686 (+ 87us) server_negotiation.cc:407] Connection header received
0925 14:49:07.578788 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.578789 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.578798 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.578851 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.579383 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.579927 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.580659 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.580870 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.580950 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.582085 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.582087 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.582087 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:07.582094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.583327 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.583662 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.583663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.583663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.583714 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:07.583960 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:07.583961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:07.583961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:07.584102 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.584209 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.584313 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.584392 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 14:49:07.586180 7152 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:07.587865 7152 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:07.594158 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.589974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:45269 (local address 127.0.0.1:59536)
0925 14:49:07.590118 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.590158 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.590218 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.590519 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.590521 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.590523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:07.590568 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.590569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.591590 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.591591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.592908 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.592910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.593847 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.593848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.593929 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.593942 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.594022 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.594076 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":53,"threads_started":1}
I20260925 14:49:07.594177 7628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.590074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.21.205.193:45269)
0925 14:49:07.590342 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.590346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.590357 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:07.590395 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.590397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.590412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.590518 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:07.590658 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.591494 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.592997 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.593744 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.593822 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.593876 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.594051 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.594094 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":90,"threads_started":1}
I20260925 14:49:07.594313 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.589995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:44393 (local address 127.0.0.1:48286)
0925 14:49:07.590247 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.590258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.590313 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.590900 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.590901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.590904 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:07.590942 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.590943 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.592261 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.592262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.593230 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.593232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.594080 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.594082 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.594141 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.594145 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.594176 (+ 31us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.594220 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":71,"threads_started":1}
I20260925 14:49:07.594419 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.590164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:42249 (local address 127.0.0.1:54978)
0925 14:49:07.590365 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.590374 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.590420 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.590734 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.590735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.590737 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:07.590790 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.590792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.592081 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.592082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.593146 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.593148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.594118 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.594119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.594176 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.594180 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.594230 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.594282 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":68,"threads_started":1}
I20260925 14:49:07.594659 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.590151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48286 (local address 127.21.205.194:44393)
0925 14:49:07.590736 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.590741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.590750 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:07.590787 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.590789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.590805 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.591033 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:07.591104 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.592158 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.593298 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.593968 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.594416 (+ 448us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.594478 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.594505 (+ 27us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.594548 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":70,"threads_started":1}
I20260925 14:49:07.594470 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.590348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54978 (local address 127.21.205.195:42249)
0925 14:49:07.590521 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.590525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.590536 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:07.590570 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.590572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.590586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.590658 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:07.590871 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.591979 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.593247 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.593971 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.594034 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.594084 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.594357 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.594395 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":80,"threads_started":1}
I20260925 14:49:07.595275 7279 tablet_service.cc:1467] Processing CreateTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 (DEFAULT_TABLE table=test-table [id=a4d819c192ec4e239adcadd384ea71c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:07.595405 7549 tablet_service.cc:1467] Processing CreateTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 (DEFAULT_TABLE table=test-table [id=a4d819c192ec4e239adcadd384ea71c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:07.595405 7414 tablet_service.cc:1467] Processing CreateTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 (DEFAULT_TABLE table=test-table [id=a4d819c192ec4e239adcadd384ea71c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:07.595541 7279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e65c7dc2af4e97a19869f3a73749d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:07.595705 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e65c7dc2af4e97a19869f3a73749d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:07.595705 7549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28e65c7dc2af4e97a19869f3a73749d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:07.598170 7632 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: Bootstrap starting.
I20260925 14:49:07.598739 7632 tablet_bootstrap.cc:654] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:07.598908 7634 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Bootstrap starting.
I20260925 14:49:07.599345 7632 log.cc:824] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:07.599560 7634 tablet_bootstrap.cc:654] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:07.600142 7634 log.cc:824] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:07.600179 7632 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: No bootstrap required, opened a new log
I20260925 14:49:07.600234 7632 ts_tablet_manager.cc:1397] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:49:07.600476 7633 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Bootstrap starting.
I20260925 14:49:07.600831 7634 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: No bootstrap required, opened a new log
I20260925 14:49:07.600910 7634 ts_tablet_manager.cc:1397] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:07.601114 7633 tablet_bootstrap.cc:654] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:07.601691 7633 log.cc:824] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:07.602438 7633 tablet_bootstrap.cc:492] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: No bootstrap required, opened a new log
I20260925 14:49:07.602504 7633 ts_tablet_manager.cc:1397] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:49:07.602384 7632 raft_consensus.cc:348] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.602562 7632 raft_consensus.cc:374] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:07.602627 7632 raft_consensus.cc:729] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b66bb7d960431cbbb83e18db78ee2b, State: Initialized, Role: FOLLOWER
I20260925 14:49:07.602751 7632 consensus_queue.cc:260] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.602777 7634 raft_consensus.cc:348] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.602972 7634 raft_consensus.cc:374] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:07.603019 7632 ts_tablet_manager.cc:1428] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:49:07.603017 7634 raft_consensus.cc:729] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8952875df5c444f8f0543b9c6a03ed2, State: Initialized, Role: FOLLOWER
I20260925 14:49:07.603138 7634 consensus_queue.cc:260] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.603178 7345 heartbeater.cc:503] Master 127.21.205.254:41113 was elected leader, sending a full tablet report...
I20260925 14:49:07.603394 7634 ts_tablet_manager.cc:1428] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 14:49:07.603518 7481 heartbeater.cc:503] Master 127.21.205.254:41113 was elected leader, sending a full tablet report...
I20260925 14:49:07.604821 7633 raft_consensus.cc:348] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.604964 7633 raft_consensus.cc:374] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:07.605007 7633 raft_consensus.cc:729] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70a756e2f8240a4bd99136f9c89cc88, State: Initialized, Role: FOLLOWER
I20260925 14:49:07.605108 7633 consensus_queue.cc:260] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.605370 7633 ts_tablet_manager.cc:1428] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 14:49:07.605392 7615 heartbeater.cc:503] Master 127.21.205.254:41113 was elected leader, sending a full tablet report...
I20260925 14:49:07.636520 7639 raft_consensus.cc:482] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:07.636629 7639 raft_consensus.cc:504] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.637043 7639 leader_election.cc:288] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5b66bb7d960431cbbb83e18db78ee2b (127.21.205.193:45269), f70a756e2f8240a4bd99136f9c89cc88 (127.21.205.195:42249)
I20260925 14:49:07.639925 7631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.637270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:41883 (local address 127.21.205.195:42249)
0925 14:49:07.637328 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.637330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.637476 (+ 146us) server_negotiation.cc:407] Connection header received
0925 14:49:07.637546 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.637547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.637557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.637617 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:07.637805 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.638244 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639200 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.639550 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639606 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.639634 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.639813 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.639858 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260925 14:49:07.639930 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.637207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:42249 (local address 127.21.205.194:41883)
0925 14:49:07.637400 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.637411 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.637469 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.637642 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.637643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.637661 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:07.637736 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.637738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.638363 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.638364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.639094 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.639096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.639634 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.639708 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.639711 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.639762 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.639841 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":116,"threads_started":1}
I20260925 14:49:07.639971 7628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.637315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59649 (local address 127.21.205.193:45269)
0925 14:49:07.637364 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.637366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.637375 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:07.637403 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.637403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.637416 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.637478 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:07.637692 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.638219 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639159 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.639551 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639646 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.639654 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.639858 (+ 204us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.639896 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260925 14:49:07.640190 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.637194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:45269 (local address 127.21.205.194:59649)
0925 14:49:07.637267 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.637281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:07.637333 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:07.637531 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:07.637532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:07.637534 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:07.637612 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.637614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.638306 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.638307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.639068 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:07.639070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.639664 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.639665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:07.639730 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.639733 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:07.639849 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 14:49:07.639884 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":28}
I20260925 14:49:07.640318 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e65c7dc2af4e97a19869f3a73749d4" candidate_uuid: "f8952875df5c444f8f0543b9c6a03ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70a756e2f8240a4bd99136f9c89cc88" is_pre_election: true
I20260925 14:49:07.640476 7569 raft_consensus.cc:2454] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8952875df5c444f8f0543b9c6a03ed2 in term 0.
I20260925 14:49:07.640514 7299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e65c7dc2af4e97a19869f3a73749d4" candidate_uuid: "f8952875df5c444f8f0543b9c6a03ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" is_pre_election: true
I20260925 14:49:07.640671 7299 raft_consensus.cc:2454] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8952875df5c444f8f0543b9c6a03ed2 in term 0.
I20260925 14:49:07.640697 7366 leader_election.cc:302] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f70a756e2f8240a4bd99136f9c89cc88, f8952875df5c444f8f0543b9c6a03ed2; no voters:
I20260925 14:49:07.640811 7639 raft_consensus.cc:2781] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:07.640878 7639 raft_consensus.cc:482] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:07.640903 7639 raft_consensus.cc:3037] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:07.641566 7639 raft_consensus.cc:504] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.641705 7639 leader_election.cc:288] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [CANDIDATE]: Term 1 election: Requested vote from peers b5b66bb7d960431cbbb83e18db78ee2b (127.21.205.193:45269), f70a756e2f8240a4bd99136f9c89cc88 (127.21.205.195:42249)
I20260925 14:49:07.641841 7299 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e65c7dc2af4e97a19869f3a73749d4" candidate_uuid: "f8952875df5c444f8f0543b9c6a03ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b66bb7d960431cbbb83e18db78ee2b"
I20260925 14:49:07.641873 7569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28e65c7dc2af4e97a19869f3a73749d4" candidate_uuid: "f8952875df5c444f8f0543b9c6a03ed2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f70a756e2f8240a4bd99136f9c89cc88"
I20260925 14:49:07.641916 7299 raft_consensus.cc:3037] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:07.641948 7569 raft_consensus.cc:3037] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:07.642619 7569 raft_consensus.cc:2454] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8952875df5c444f8f0543b9c6a03ed2 in term 1.
I20260925 14:49:07.642620 7299 raft_consensus.cc:2454] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8952875df5c444f8f0543b9c6a03ed2 in term 1.
I20260925 14:49:07.642779 7366 leader_election.cc:302] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f70a756e2f8240a4bd99136f9c89cc88, f8952875df5c444f8f0543b9c6a03ed2; no voters:
I20260925 14:49:07.642865 7639 raft_consensus.cc:2781] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:07.643042 7639 raft_consensus.cc:686] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 1 LEADER]: Becoming Leader. State: Replica: f8952875df5c444f8f0543b9c6a03ed2, State: Running, Role: LEADER
I20260925 14:49:07.643152 7639 consensus_queue.cc:237] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } }
I20260925 14:49:07.644013 7152 catalog_manager.cc:5495] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 reported cstate change: term changed from 0 to 1, leader changed from <none> to f8952875df5c444f8f0543b9c6a03ed2 (127.21.205.194). New cstate: current_term: 1 leader_uuid: "f8952875df5c444f8f0543b9c6a03ed2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8952875df5c444f8f0543b9c6a03ed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 44393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 } health_report { overall_health: UNKNOWN } } }
I20260925 14:49:07.650324 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.648185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48288 (local address 127.21.205.194:44393)
0925 14:49:07.648219 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.648221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.648229 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:07.648255 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.648256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.648270 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.648325 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:07.648407 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.648848 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.649626 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.649829 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.649903 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.650046 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:07.650136 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:07.650202 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.650239 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.650257 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:49:07.652033 7299 raft_consensus.cc:1261] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 1 FOLLOWER]: Refusing update from remote peer f8952875df5c444f8f0543b9c6a03ed2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:07.652108 7569 raft_consensus.cc:1261] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 1 FOLLOWER]: Refusing update from remote peer f8952875df5c444f8f0543b9c6a03ed2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:07.652388 7639 consensus_queue.cc:1035] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b66bb7d960431cbbb83e18db78ee2b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 45269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:07.652544 7642 consensus_queue.cc:1035] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f70a756e2f8240a4bd99136f9c89cc88" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 42249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:07.655678 7647 mvcc.cc:205] Tried to move back new op lower bound from 7333264374380519424 to 7333264374347161600. Current Snapshot: MvccSnapshot[applied={T|T < 7333264374380519424 or (T in {7333264374380519424})}]
I20260925 14:49:07.657162 7153 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:07.657256 7153 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:07.657974 7153 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49bfcaa49b6c44d8af031ab27ba7db4b: Sending DeleteTablet for 3 replicas of tablet 28e65c7dc2af4e97a19869f3a73749d4
I20260925 14:49:07.658288 7279 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:07 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:59536
I20260925 14:49:07.658319 7549 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:07 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:54978
I20260925 14:49:07.658519 7655 tablet_replica.cc:330] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: stopping tablet replica
I20260925 14:49:07.658545 7414 tablet_service.cc:1514] Processing DeleteTablet for tablet 28e65c7dc2af4e97a19869f3a73749d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:07 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:48286
I20260925 14:49:07.658649 7655 raft_consensus.cc:2229] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:07.658735 7654 tablet_replica.cc:330] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b: stopping tablet replica
I20260925 14:49:07.658744 7655 raft_consensus.cc:2258] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:07.658820 7654 raft_consensus.cc:2229] T 28e65c7dc2af4e97a19869f3a73749d4 P b5b66bb7d960431cbbb83e18db78ee2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:07.658993 7656 tablet_replica.cc:330] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: stopping tablet replica
I20260925 14:49:07.659096 7656 raft_consensus.cc:2229] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:49:07.659114 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7215
I20260925 14:49:07.659168 7655 ts_tablet_manager.cc:1905] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:07.659204 7656 raft_consensus.cc:2258] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:07.661839 7656 ts_tablet_manager.cc:1905] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:07.664119 7655 ts_tablet_manager.cc:1918] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:07.664201 7655 log.cc:1196] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/28e65c7dc2af4e97a19869f3a73749d4
I20260925 14:49:07.664528 7655 ts_tablet_manager.cc:1939] T 28e65c7dc2af4e97a19869f3a73749d4 P f70a756e2f8240a4bd99136f9c89cc88: Deleting consensus metadata
I20260925 14:49:07.664951 7137 catalog_manager.cc:4841] TS f70a756e2f8240a4bd99136f9c89cc88 (127.21.205.195:42249): tablet 28e65c7dc2af4e97a19869f3a73749d4 (table test-table [id=a4d819c192ec4e239adcadd384ea71c4]) successfully deleted
I20260925 14:49:07.666450 7656 ts_tablet_manager.cc:1918] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:07.666523 7656 log.cc:1196] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/28e65c7dc2af4e97a19869f3a73749d4
I20260925 14:49:07.666792 7656 ts_tablet_manager.cc:1939] T 28e65c7dc2af4e97a19869f3a73749d4 P f8952875df5c444f8f0543b9c6a03ed2: Deleting consensus metadata
W20260925 14:49:07.667356 7138 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:45269 (error 108)
I20260925 14:49:07.667544 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7350
I20260925 14:49:07.668614 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.667703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:45269 (local address 127.0.0.1:59544)
0925 14:49:07.668536 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.668578 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":812}
W20260925 14:49:07.668772 7138 catalog_manager.cc:4584] TS b5b66bb7d960431cbbb83e18db78ee2b (127.21.205.193:45269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 28e65c7dc2af4e97a19869f3a73749d4: Network error: Client connection negotiation failed: client connection to 127.21.205.193:45269: connect: Connection refused (error 111)
I20260925 14:49:07.675411 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7486
I20260925 14:49:07.675498 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.675367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:44393 (local address 127.0.0.1:48290)
0925 14:49:07.675416 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:07.675447 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.194:44393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 14:49:07.679824 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7120
2026-09-25T14:49:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1886 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T14:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:07Z Disabled control of system clock
I20260925 14:49:07.707082 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:07.793006 7661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.793223 7661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.793249 7661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.794323 7661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:07.794363 7661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.794378 7661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.794392 7661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:07.794406 7661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:07.795706 7661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:32805
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.795919 7661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.796176 7661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.800944 7666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.801127 7661 server_base.cc:1034] running on GCE node
W20260925 14:49:07.800956 7669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.800956 7667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.801479 7661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.801735 7661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.802901 7661 hybrid_clock.cc:648] HybridClock initialized: now 1790347747802883 us; error 36 us; skew 500 ppm
I20260925 14:49:07.804111 7661 webserver.cc:460] Webserver started at http://127.21.205.254:33913/ using document root <none> and password file <none>
I20260925 14:49:07.804349 7661 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.804417 7661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.804572 7661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.805559 7661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "2f8f85fa8dd640ef802984b54c45c083"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b4dde650698700ae4a19e85d02843b47"
server_key_iv: "e41de4b79422a2c22e69899779719d0c"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.805913 7661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "2f8f85fa8dd640ef802984b54c45c083"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b4dde650698700ae4a19e85d02843b47"
server_key_iv: "e41de4b79422a2c22e69899779719d0c"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.807170 7661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:07.807874 7675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.808050 7661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:07.808153 7661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "2f8f85fa8dd640ef802984b54c45c083"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "b4dde650698700ae4a19e85d02843b47"
server_key_iv: "e41de4b79422a2c22e69899779719d0c"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.808240 7661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:07.844890 7661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:07.845208 7661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:07.845366 7661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:07.849362 7661 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:32805
I20260925 14:49:07.849395 7729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:32805 every 8 connection(s)
I20260925 14:49:07.849717 7661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:07.850642 7730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:07.852828 7730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: Bootstrap starting.
I20260925 14:49:07.853410 7730 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:07.853701 7730 log.cc:824] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:07.854343 7730 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: No bootstrap required, opened a new log
I20260925 14:49:07.855878 7730 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:49:07.856004 7730 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:07.856025 7730 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f8f85fa8dd640ef802984b54c45c083, State: Initialized, Role: FOLLOWER
I20260925 14:49:07.856195 7730 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:49:07.856263 7730 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:07.856307 7730 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:07.856359 7730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:07.856828 7730 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:49:07.856936 7730 leader_election.cc:302] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f8f85fa8dd640ef802984b54c45c083; no voters:
I20260925 14:49:07.857126 7730 leader_election.cc:288] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:07.857161 7733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:07.857195 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7661
I20260925 14:49:07.857285 7733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [term 1 LEADER]: Becoming Leader. State: Replica: 2f8f85fa8dd640ef802984b54c45c083, State: Running, Role: LEADER
I20260925 14:49:07.857300 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:07.857383 7730 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:07.857365 7733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } }
I20260925 14:49:07.857499 22327 external_mini_cluster.cc:1428] Setting key 9ef7cc7a43ad2a846033c27728ae116d
I20260925 14:49:07.857661 7733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f8f85fa8dd640ef802984b54c45c083. Latest consensus state: current_term: 1 leader_uuid: "2f8f85fa8dd640ef802984b54c45c083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:49:07.858109 7733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:07.858089 7735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f8f85fa8dd640ef802984b54c45c083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f8f85fa8dd640ef802984b54c45c083" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32805 } } }
I20260925 14:49:07.858372 7735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:07.858496 7744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:07.859061 7744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:07.860670 7744 catalog_manager.cc:1347] Generated new cluster ID: 9bcc453a3a124fefab88f2185b12dbc3
I20260925 14:49:07.860720 7744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:07.863376 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:07.857930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37290 (local address 127.21.205.254:32805)
0925 14:49:07.858578 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:07.858587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:07.858884 (+ 297us) server_negotiation.cc:407] Connection header received
0925 14:49:07.859111 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:07.859121 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:07.859217 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:07.859381 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:07.859607 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.860628 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.861427 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:07.861943 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:07.862081 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:07.862727 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:07.862730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:07.862730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:07.862901 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:07.862930 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:07.862941 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:07.863011 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:07.863082 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 14:49:07.863157 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":75,"threads_started":1}
I20260925 14:49:07.878360 7744 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:07.878871 7744 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:07.881695 7744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: Generated new TSK 0
I20260925 14:49:07.881868 7744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:07.888782 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:07.978843 7754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:07.979038 7754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:07.979058 7754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:07.980239 7754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:07.980286 7754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:07.980311 7754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:07.981621 7754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:07.981846 7754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:07.982139 7754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:07.987967 7759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.988085 7754 server_base.cc:1034] running on GCE node
W20260925 14:49:07.988013 7760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:07.988237 7762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:07.988467 7754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:07.988740 7754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:07.989898 7754 hybrid_clock.cc:648] HybridClock initialized: now 1790347747989884 us; error 30 us; skew 500 ppm
I20260925 14:49:07.991134 7754 webserver.cc:460] Webserver started at http://127.21.205.193:46609/ using document root <none> and password file <none>
I20260925 14:49:07.991391 7754 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:07.991436 7754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:07.991586 7754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:07.992693 7754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "d1b0afd7a34c4e5d9039265731ed9883"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "37d095c5e17790f52973670db07a3bc8"
server_key_iv: "f0a638020a6180461782cf860b1bdbb7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.993044 7754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "d1b0afd7a34c4e5d9039265731ed9883"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "37d095c5e17790f52973670db07a3bc8"
server_key_iv: "f0a638020a6180461782cf860b1bdbb7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.994405 7754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:07.995173 7768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:07.995357 7754 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:07.995446 7754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "d1b0afd7a34c4e5d9039265731ed9883"
format_stamp: "Formatted at 2026-09-25 14:49:07 on dist-test-slave-d7lw"
server_key: "37d095c5e17790f52973670db07a3bc8"
server_key_iv: "f0a638020a6180461782cf860b1bdbb7"
server_key_version: "encryptionkey@0"
I20260925 14:49:07.995535 7754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:08.012552 7754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:08.012844 7754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:08.012989 7754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:08.013211 7754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:08.013528 7754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:08.013581 7754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.013621 7754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:08.013649 7754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.018929 7754 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:38431
I20260925 14:49:08.019012 7883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:38431 every 8 connection(s)
I20260925 14:49:08.019305 7754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:49:08.024312 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7754
I20260925 14:49:08.024431 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:08.024528 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.020026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:43611 (local address 127.21.205.254:32805)
0925 14:49:08.020076 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.020079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.020287 (+ 208us) server_negotiation.cc:407] Connection header received
0925 14:49:08.020823 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.020824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.020838 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.020888 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:08.021524 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.022049 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.023032 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.023254 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.023331 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.024243 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:08.024244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:08.024245 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:08.024249 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:08.024263 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:08.024266 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:08.024320 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.024418 (+ 98us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.024450 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:49:08.024677 22327 external_mini_cluster.cc:1428] Setting key 1dfabfefcb5dbadf03594d279a5011e2
I20260925 14:49:08.024794 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.019999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.193:43611)
0925 14:49:08.020171 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.020202 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.020705 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.020954 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.020958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.020967 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:08.021430 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.021435 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.022167 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.022168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.022936 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.022938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.023341 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.023342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.023480 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.024082 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:08.024082 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:08.024111 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:08.024133 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:08.024323 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:08.024329 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:08.024331 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.024419 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.024516 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260925 14:49:08.025266 7884 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:49:08.025400 7884 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:08.025579 7884 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:49:08.025676 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 14:49:08.026158 7693 ts_manager.cc:194] Registered new tserver with Master: d1b0afd7a34c4e5d9039265731ed9883 (127.21.205.193:38431)
I20260925 14:49:08.026774 7693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.193:43611
W20260925 14:49:08.106041 7887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:08.106237 7887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:08.106266 7887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:08.107473 7887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:08.107515 7887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:08.107540 7887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:08.108899 7887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:08.109135 7887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:08.109429 7887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:08.114456 7892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:08.114493 7893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:08.114617 7887 server_base.cc:1034] running on GCE node
W20260925 14:49:08.114503 7895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:08.115024 7887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:08.115296 7887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:08.116456 7887 hybrid_clock.cc:648] HybridClock initialized: now 1790347748116447 us; error 39 us; skew 500 ppm
I20260925 14:49:08.117772 7887 webserver.cc:460] Webserver started at http://127.21.205.194:37481/ using document root <none> and password file <none>
I20260925 14:49:08.118011 7887 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:08.118091 7887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:08.118237 7887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:08.119318 7887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "51f9efa1bb22c11024ed7dead1601d1a"
server_key_iv: "0a51097605b517a19ccc9c2b98cf6f18"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.119695 7887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "51f9efa1bb22c11024ed7dead1601d1a"
server_key_iv: "0a51097605b517a19ccc9c2b98cf6f18"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.120990 7887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:08.121690 7901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.121855 7887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:08.121955 7887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "51f9efa1bb22c11024ed7dead1601d1a"
server_key_iv: "0a51097605b517a19ccc9c2b98cf6f18"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.122030 7887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:08.134897 7887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:08.135202 7887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:08.135349 7887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:08.135568 7887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:08.135962 7887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:08.136016 7887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.136050 7887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:08.136080 7887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.141736 7887 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:41217
I20260925 14:49:08.141777 8015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:41217 every 8 connection(s)
I20260925 14:49:08.142095 7887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:49:08.146926 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.142785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:50169 (local address 127.21.205.254:32805)
0925 14:49:08.142839 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.142842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.143085 (+ 243us) server_negotiation.cc:407] Connection header received
0925 14:49:08.143556 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.143557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.143567 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.143615 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:08.144296 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.144815 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.145735 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.145921 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.145972 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.146702 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:08.146703 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:08.146703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:08.146707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:08.146721 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:08.146724 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:08.146767 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.146846 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.146870 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:49:08.147126 8018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.142782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.194:50169)
0925 14:49:08.142968 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.143000 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.143472 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.143677 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.143679 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.143687 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:08.144185 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.144189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.144894 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.144895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.145638 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.145640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.145993 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.145994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.146097 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.146577 (+ 480us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:08.146577 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:08.146604 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:08.146625 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:08.146775 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:08.146779 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:08.146781 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.146831 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.146905 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260925 14:49:08.147554 8016 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:49:08.147665 8016 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:08.147850 8016 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:49:08.148195 7693 ts_manager.cc:194] Registered new tserver with Master: e7ca501eebe7483b9dc1b2eb8bf6025e (127.21.205.194:41217)
I20260925 14:49:08.148559 7693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.194:50169
I20260925 14:49:08.150470 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 7887
I20260925 14:49:08.150594 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:08.150794 22327 external_mini_cluster.cc:1428] Setting key 7bd3c58b9108eb3a0ec757c0fb4a3730
I20260925 14:49:08.151603 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--builtin_ntp_servers=127.21.205.212:40159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:08.232753 8019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:08.232954 8019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:08.232977 8019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:08.234130 8019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:08.234172 8019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:08.234221 8019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:08.235584 8019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:40159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:08.235843 8019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:08.236150 8019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:08.241050 8024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:08.241065 8025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:08.241267 8019 server_base.cc:1034] running on GCE node
W20260925 14:49:08.241281 8027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:08.241577 8019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:08.241827 8019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:08.242985 8019 hybrid_clock.cc:648] HybridClock initialized: now 1790347748242975 us; error 36 us; skew 500 ppm
I20260925 14:49:08.244302 8019 webserver.cc:460] Webserver started at http://127.21.205.195:41763/ using document root <none> and password file <none>
I20260925 14:49:08.244571 8019 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:08.244644 8019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:08.244797 8019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:08.245844 8019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "585a5883ff3d49e7a69b882c555b48c0"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "0787a10470a202a98543e6ba4bdefca7"
server_key_iv: "3b6ecc4e6999f4df3a31f8185bff12f9"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.246212 8019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "585a5883ff3d49e7a69b882c555b48c0"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "0787a10470a202a98543e6ba4bdefca7"
server_key_iv: "3b6ecc4e6999f4df3a31f8185bff12f9"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.247490 8019 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:49:08.248265 8033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.248436 8019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.248533 8019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "585a5883ff3d49e7a69b882c555b48c0"
format_stamp: "Formatted at 2026-09-25 14:49:08 on dist-test-slave-d7lw"
server_key: "0787a10470a202a98543e6ba4bdefca7"
server_key_iv: "3b6ecc4e6999f4df3a31f8185bff12f9"
server_key_version: "encryptionkey@0"
I20260925 14:49:08.248628 8019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:08.266139 8019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:08.266436 8019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:08.266588 8019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:08.266808 8019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:08.267125 8019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:08.267179 8019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.267220 8019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:08.267248 8019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:08.272917 8019 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:37649
I20260925 14:49:08.272964 8147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:37649 every 8 connection(s)
I20260925 14:49:08.273277 8019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:49:08.276350 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8019
I20260925 14:49:08.276429 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:08.276743 22327 external_mini_cluster.cc:1428] Setting key 2dad8b2e5a882883af69cc9061f4d68d
I20260925 14:49:08.278777 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.273926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:45559 (local address 127.21.205.254:32805)
0925 14:49:08.273997 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.274000 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.274222 (+ 222us) server_negotiation.cc:407] Connection header received
0925 14:49:08.274727 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.274729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.274744 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.274802 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:08.275473 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.276066 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.277367 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.277688 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.277753 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.278527 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:08.278528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:08.278528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:08.278538 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:08.278551 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:08.278579 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:08.278624 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.278690 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.278718 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:49:08.279006 8150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.273926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32805 (local address 127.21.205.195:45559)
0925 14:49:08.274118 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.274150 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.274615 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.274875 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.274878 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.274886 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:08.275381 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.275385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.276147 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.276148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.277287 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.277290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.277775 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.277776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.277897 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.278402 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:08.278402 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:08.278430 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:08.278451 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:08.278632 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:08.278636 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:08.278638 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.278685 (+ 47us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.278763 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260925 14:49:08.279469 8148 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32805
I20260925 14:49:08.279569 8148 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:08.279744 8148 heartbeater.cc:511] Master 127.21.205.254:32805 requested a full tablet report, sending...
I20260925 14:49:08.280146 7693 ts_manager.cc:194] Registered new tserver with Master: 585a5883ff3d49e7a69b882c555b48c0 (127.21.205.195:37649)
I20260925 14:49:08.280504 7693 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.195:45559
I20260925 14:49:08.288465 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:08.293015 7743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.289887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.21.205.254:32805)
0925 14:49:08.289978 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.289981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.290075 (+ 94us) server_negotiation.cc:407] Connection header received
0925 14:49:08.290194 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.290195 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.290206 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.290255 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:08.290427 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.290960 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.291815 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.292040 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.292126 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.292750 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:08.292752 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:08.292752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:08.292757 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:08.292771 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:08.292775 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:08.292834 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.292902 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.292924 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 14:49:08.294476 7693 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:08.296170 7693 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:08.302731 8160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:37649 (local address 127.0.0.1:53076)
0925 14:49:08.298931 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.298943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.299014 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.299255 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.299256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.299258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:08.299299 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.299300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300274 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.300275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.301635 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.301638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.302413 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.302414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.302480 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.302500 (+ 20us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.302592 (+ 92us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.302654 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":31,"thread_start_us":85,"threads_started":1}
I20260925 14:49:08.302820 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53076 (local address 127.21.205.195:37649)
0925 14:49:08.298901 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.298906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.299061 (+ 155us) server_negotiation.cc:407] Connection header received
0925 14:49:08.299106 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.299108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.299123 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.299194 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:08.299353 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300178 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.301728 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.302301 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.302404 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.302495 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.302679 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.302718 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260925 14:49:08.303925 8156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:38431 (local address 127.0.0.1:39546)
0925 14:49:08.298443 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.298481 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.298586 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.298764 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.298767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.298768 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:08.298830 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.298832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300331 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.300332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.301635 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.301638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.303645 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.303646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.303712 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.303715 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.303772 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.303838 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":60,"thread_start_us":57,"threads_started":1}
I20260925 14:49:08.303848 8081 tablet_service.cc:1467] Processing CreateTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 (DEFAULT_TABLE table=test-table [id=4127d2bc6c8e43ee888f952420bdefa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:08.304178 8081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f00a4f67a904f13bb41491bb85b4043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:08.304337 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39546 (local address 127.21.205.193:38431)
0925 14:49:08.298558 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.298563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.298574 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:08.298666 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.298668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.298682 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.298760 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:08.298929 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300261 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.302535 (+ 2274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.303442 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.303510 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.303600 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.304189 (+ 589us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.304244 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260925 14:49:08.305105 7817 tablet_service.cc:1467] Processing CreateTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 (DEFAULT_TABLE table=test-table [id=4127d2bc6c8e43ee888f952420bdefa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:08.305245 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.21.205.194:41217)
0925 14:49:08.299150 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.299155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.299167 (+ 12us) server_negotiation.cc:407] Connection header received
0925 14:49:08.299207 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.299209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.299225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.299305 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:08.299502 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300643 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.301823 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.302293 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.302352 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.302451 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.305133 (+ 2682us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.305178 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":77,"threads_started":1}
I20260925 14:49:08.305435 7817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f00a4f67a904f13bb41491bb85b4043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:08.305519 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.298278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:41217 (local address 127.0.0.1:56496)
0925 14:49:08.298553 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.298564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.298657 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.299372 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.299373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.299375 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:08.299425 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.299427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.300723 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.300724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.301737 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.301740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.302389 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.302390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.302477 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.302491 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.302617 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.305421 (+ 2804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
I20260925 14:49:08.306154 7949 tablet_service.cc:1467] Processing CreateTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 (DEFAULT_TABLE table=test-table [id=4127d2bc6c8e43ee888f952420bdefa2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:08.306471 7949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f00a4f67a904f13bb41491bb85b4043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:08.307633 8162 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Bootstrap starting.
I20260925 14:49:08.308411 8162 tablet_bootstrap.cc:654] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:08.308774 8163 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: Bootstrap starting.
I20260925 14:49:08.308773 8162 log.cc:824] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:08.309414 8163 tablet_bootstrap.cc:654] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:08.309619 8162 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: No bootstrap required, opened a new log
I20260925 14:49:08.309696 8165 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Bootstrap starting.
I20260925 14:49:08.309773 8162 ts_tablet_manager.cc:1397] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:08.309823 8163 log.cc:824] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:08.310348 8165 tablet_bootstrap.cc:654] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:08.310674 8163 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: No bootstrap required, opened a new log
I20260925 14:49:08.310655 8165 log.cc:824] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:08.310738 8163 ts_tablet_manager.cc:1397] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:49:08.312412 8165 tablet_bootstrap.cc:492] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: No bootstrap required, opened a new log
I20260925 14:49:08.312484 8165 ts_tablet_manager.cc:1397] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 14:49:08.312855 8162 raft_consensus.cc:348] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.313016 8162 raft_consensus.cc:374] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:08.313061 8162 raft_consensus.cc:729] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 585a5883ff3d49e7a69b882c555b48c0, State: Initialized, Role: FOLLOWER
I20260925 14:49:08.313182 8162 consensus_queue.cc:260] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.313470 8162 ts_tablet_manager.cc:1428] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 14:49:08.313397 8163 raft_consensus.cc:348] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.313540 8163 raft_consensus.cc:374] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:08.313580 8163 raft_consensus.cc:729] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1b0afd7a34c4e5d9039265731ed9883, State: Initialized, Role: FOLLOWER
I20260925 14:49:08.313622 8148 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
I20260925 14:49:08.313690 8163 consensus_queue.cc:260] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.313936 8163 ts_tablet_manager.cc:1428] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:49:08.314052 7884 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
I20260925 14:49:08.315054 8165 raft_consensus.cc:348] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.315191 8165 raft_consensus.cc:374] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:08.315212 8165 raft_consensus.cc:729] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7ca501eebe7483b9dc1b2eb8bf6025e, State: Initialized, Role: FOLLOWER
I20260925 14:49:08.315316 8165 consensus_queue.cc:260] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.315565 8165 ts_tablet_manager.cc:1428] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:49:08.315626 8016 heartbeater.cc:503] Master 127.21.205.254:32805 was elected leader, sending a full tablet report...
W20260925 14:49:08.392627 8017 tablet.cc:2369] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:49:08.397050 8169 raft_consensus.cc:482] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:08.397154 8169 raft_consensus.cc:504] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.397540 8169 leader_election.cc:288] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7ca501eebe7483b9dc1b2eb8bf6025e (127.21.205.194:41217), 585a5883ff3d49e7a69b882c555b48c0 (127.21.205.195:37649)
I20260925 14:49:08.400512 7886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.397749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:41217 (local address 127.21.205.193:60307)
0925 14:49:08.397804 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.397817 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.397865 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.398028 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.398029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.398031 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:08.398108 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.398110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.398745 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.398746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.399646 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.399648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.400246 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.400246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.400320 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.400324 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.400400 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.400441 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 14:49:08.400512 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.397841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:60307 (local address 127.21.205.194:41217)
0925 14:49:08.397901 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.397903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.397914 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:08.397947 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.397947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.397957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.398020 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:08.398183 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.398662 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.399740 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.400160 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.400246 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.400262 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.400405 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.400443 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:49:08.400787 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.397897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:37649 (local address 127.21.205.193:57303)
0925 14:49:08.398125 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.398133 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:08.398200 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:08.398340 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:08.398341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:08.398342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:08.398415 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.398421 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.399040 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.399041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.399963 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:08.399965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.400529 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.400530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:08.400605 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.400611 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 14:49:08.400668 (+ 57us) client_negotiation.cc:240] Negotiation successful
0925 14:49:08.400712 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":113,"threads_started":1}
I20260925 14:49:08.400815 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.397966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:57303 (local address 127.21.205.195:37649)
0925 14:49:08.398035 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.398037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.398187 (+ 150us) server_negotiation.cc:407] Connection header received
0925 14:49:08.398257 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.398258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.398273 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.398336 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:08.398494 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.398958 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.400048 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.400436 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.400521 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.400529 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.400695 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.400744 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:49:08.400868 7969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f00a4f67a904f13bb41491bb85b4043" candidate_uuid: "d1b0afd7a34c4e5d9039265731ed9883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" is_pre_election: true
I20260925 14:49:08.401000 7969 raft_consensus.cc:2454] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b0afd7a34c4e5d9039265731ed9883 in term 0.
I20260925 14:49:08.401243 7770 leader_election.cc:302] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1b0afd7a34c4e5d9039265731ed9883, e7ca501eebe7483b9dc1b2eb8bf6025e; no voters:
I20260925 14:49:08.401353 8101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f00a4f67a904f13bb41491bb85b4043" candidate_uuid: "d1b0afd7a34c4e5d9039265731ed9883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585a5883ff3d49e7a69b882c555b48c0" is_pre_election: true
I20260925 14:49:08.401401 8169 raft_consensus.cc:2781] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:08.401445 8169 raft_consensus.cc:482] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:08.401467 8169 raft_consensus.cc:3037] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:08.401499 8101 raft_consensus.cc:2454] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1b0afd7a34c4e5d9039265731ed9883 in term 0.
I20260925 14:49:08.402220 8169 raft_consensus.cc:504] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.402352 8169 leader_election.cc:288] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [CANDIDATE]: Term 1 election: Requested vote from peers e7ca501eebe7483b9dc1b2eb8bf6025e (127.21.205.194:41217), 585a5883ff3d49e7a69b882c555b48c0 (127.21.205.195:37649)
I20260925 14:49:08.402520 8101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f00a4f67a904f13bb41491bb85b4043" candidate_uuid: "d1b0afd7a34c4e5d9039265731ed9883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "585a5883ff3d49e7a69b882c555b48c0"
I20260925 14:49:08.402516 7969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f00a4f67a904f13bb41491bb85b4043" candidate_uuid: "d1b0afd7a34c4e5d9039265731ed9883" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e"
I20260925 14:49:08.402593 8101 raft_consensus.cc:3037] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:08.402593 7969 raft_consensus.cc:3037] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:08.403455 8101 raft_consensus.cc:2454] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b0afd7a34c4e5d9039265731ed9883 in term 1.
I20260925 14:49:08.403455 7969 raft_consensus.cc:2454] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1b0afd7a34c4e5d9039265731ed9883 in term 1.
I20260925 14:49:08.403679 7770 leader_election.cc:302] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 585a5883ff3d49e7a69b882c555b48c0, d1b0afd7a34c4e5d9039265731ed9883; no voters:
I20260925 14:49:08.403810 8169 raft_consensus.cc:2781] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:08.403944 8169 raft_consensus.cc:686] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 1 LEADER]: Becoming Leader. State: Replica: d1b0afd7a34c4e5d9039265731ed9883, State: Running, Role: LEADER
I20260925 14:49:08.404155 8169 consensus_queue.cc:237] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } }
I20260925 14:49:08.405012 7692 catalog_manager.cc:5495] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1b0afd7a34c4e5d9039265731ed9883 (127.21.205.193). New cstate: current_term: 1 leader_uuid: "d1b0afd7a34c4e5d9039265731ed9883" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1b0afd7a34c4e5d9039265731ed9883" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 38431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 } health_report { overall_health: UNKNOWN } } }
I20260925 14:49:08.418392 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.416288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39562 (local address 127.21.205.193:38431)
0925 14:49:08.416337 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:08.416339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:08.416346 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:49:08.416377 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:08.416378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:08.416395 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:08.416458 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:08.416572 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.417014 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.417748 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:08.417934 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:08.417986 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:08.418115 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:08.418210 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:08.418269 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:08.418304 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 14:49:08.418321 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:49:08.420105 8101 raft_consensus.cc:1261] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 1 FOLLOWER]: Refusing update from remote peer d1b0afd7a34c4e5d9039265731ed9883: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:08.420127 7969 raft_consensus.cc:1261] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 1 FOLLOWER]: Refusing update from remote peer d1b0afd7a34c4e5d9039265731ed9883: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:08.420490 8169 consensus_queue.cc:1035] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [LEADER]: Connected to new peer: Peer: permanent_uuid: "585a5883ff3d49e7a69b882c555b48c0" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 37649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:08.420658 8172 consensus_queue.cc:1035] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7ca501eebe7483b9dc1b2eb8bf6025e" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 41217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:08.423050 8176 mvcc.cc:205] Tried to move back new op lower bound from 7333264377526079488 to 7333264377464053760. Current Snapshot: MvccSnapshot[applied={T|T < 7333264377526079488}]
I20260925 14:49:08.423333 8177 mvcc.cc:205] Tried to move back new op lower bound from 7333264377526079488 to 7333264377464053760. Current Snapshot: MvccSnapshot[applied={T|T < 7333264377526079488}]
I20260925 14:49:08.425235 7692 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:08.425328 7692 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:08.426064 7692 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f8f85fa8dd640ef802984b54c45c083: Sending DeleteTablet for 3 replicas of tablet 3f00a4f67a904f13bb41491bb85b4043
I20260925 14:49:08.426371 7949 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:08 UTC) from {username='slave'} at 127.0.0.1:56496
I20260925 14:49:08.426421 8081 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:08 UTC) from {username='slave'} at 127.0.0.1:53076
I20260925 14:49:08.426458 7817 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f00a4f67a904f13bb41491bb85b4043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:08 UTC) from {username='slave'} at 127.0.0.1:39546
I20260925 14:49:08.426596 8185 tablet_replica.cc:330] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: stopping tablet replica
I20260925 14:49:08.426693 8185 raft_consensus.cc:2229] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:08.426836 8187 tablet_replica.cc:330] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883: stopping tablet replica
I20260925 14:49:08.426926 8187 raft_consensus.cc:2229] T 3f00a4f67a904f13bb41491bb85b4043 P d1b0afd7a34c4e5d9039265731ed9883 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:49:08.427014 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7754
I20260925 14:49:08.428524 8185 raft_consensus.cc:2258] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:08.428942 8185 ts_tablet_manager.cc:1905] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:08.430393 8185 ts_tablet_manager.cc:1918] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:08.430472 8185 log.cc:1196] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/3f00a4f67a904f13bb41491bb85b4043
I20260925 14:49:08.430692 8185 ts_tablet_manager.cc:1939] T 3f00a4f67a904f13bb41491bb85b4043 P e7ca501eebe7483b9dc1b2eb8bf6025e: Deleting consensus metadata
I20260925 14:49:08.431061 7677 catalog_manager.cc:4841] TS e7ca501eebe7483b9dc1b2eb8bf6025e (127.21.205.194:41217): tablet 3f00a4f67a904f13bb41491bb85b4043 (table test-table [id=4127d2bc6c8e43ee888f952420bdefa2]) successfully deleted
I20260925 14:49:08.432021 8186 tablet_replica.cc:330] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: stopping tablet replica
I20260925 14:49:08.432149 8186 raft_consensus.cc:2229] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:08.432274 8186 raft_consensus.cc:2258] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:08.432735 8186 ts_tablet_manager.cc:1905] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 14:49:08.432794 7678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:38431 (error 108)
I20260925 14:49:08.433220 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:08.433106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:38431 (local address 127.0.0.1:39564)
0925 14:49:08.433157 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:08.433194 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:38431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 14:49:08.433339 7678 catalog_manager.cc:4584] TS d1b0afd7a34c4e5d9039265731ed9883 (127.21.205.193:38431): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f00a4f67a904f13bb41491bb85b4043: Network error: Client connection negotiation failed: client connection to 127.21.205.193:38431: connect: Connection refused (error 111)
I20260925 14:49:08.433591 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7887
I20260925 14:49:08.434358 8186 ts_tablet_manager.cc:1918] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:08.434451 8186 log.cc:1196] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/3f00a4f67a904f13bb41491bb85b4043
I20260925 14:49:08.434691 8186 ts_tablet_manager.cc:1939] T 3f00a4f67a904f13bb41491bb85b4043 P 585a5883ff3d49e7a69b882c555b48c0: Deleting consensus metadata
I20260925 14:49:08.435098 7677 catalog_manager.cc:4841] TS 585a5883ff3d49e7a69b882c555b48c0 (127.21.205.195:37649): tablet 3f00a4f67a904f13bb41491bb85b4043 (table test-table [id=4127d2bc6c8e43ee888f952420bdefa2]) successfully deleted
I20260925 14:49:08.439493 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8019
I20260925 14:49:08.444674 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 7661
2026-09-25T14:49:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (763 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:08 dist-test-slave-d7lw krb5kdc[8191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:08 dist-test-slave-d7lw krb5kdc[8191](info): set up 2 sockets
Sep 25 14:49:08 dist-test-slave-d7lw krb5kdc[8191](info): commencing operation
krb5kdc: starting...
W20260925 14:49:09.502416 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:09 dist-test-slave-d7lw krb5kdc[8191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:09.626374 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:45675
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:45675
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:09.707899 8204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:09.708159 8204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:09.708199 8204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:09.709394 8204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:09.709465 8204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:09.709491 8204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:09.709512 8204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:09.709540 8204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:09.711058 8204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:45675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:45675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:09.711359 8204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:09.711632 8204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:09.717082 8212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:09.717090 8210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:09.717181 8204 server_base.cc:1034] running on GCE node
W20260925 14:49:09.717338 8209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:09.717757 8204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:09.718043 8204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:09.719187 8204 hybrid_clock.cc:648] HybridClock initialized: now 1790347749719164 us; error 39 us; skew 500 ppm
Sep 25 14:49:09 dist-test-slave-d7lw krb5kdc[8191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:09.725939 8204 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:09.726476 8204 webserver.cc:460] Webserver started at http://127.21.205.254:37011/ using document root <none> and password file <none>
I20260925 14:49:09.726729 8204 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:09.726776 8204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:09.726939 8204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:09.728367 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "e51e23e6933842148fc2d2847730cc4c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "24259c44be3143c6d9d5a6b5099f9419"
server_key_iv: "27a42b4c8613fbee1cba6643d603c0b6"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.728706 8204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "e51e23e6933842148fc2d2847730cc4c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "24259c44be3143c6d9d5a6b5099f9419"
server_key_iv: "27a42b4c8613fbee1cba6643d603c0b6"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.730054 8204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:09.730849 8219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:09.731057 8204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:09.731170 8204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "e51e23e6933842148fc2d2847730cc4c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "24259c44be3143c6d9d5a6b5099f9419"
server_key_iv: "27a42b4c8613fbee1cba6643d603c0b6"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.731266 8204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:09.763485 8204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:09.766609 8204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:09.766857 8204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:09.772236 8204 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:45675
I20260925 14:49:09.772300 8271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:45675 every 8 connection(s)
I20260925 14:49:09.772754 8204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:09.773778 8272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:09.776328 8272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: Bootstrap starting.
I20260925 14:49:09.776942 8272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:09.777235 8272 log.cc:824] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:09.777969 8272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: No bootstrap required, opened a new log
I20260925 14:49:09.779588 8272 raft_consensus.cc:348] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } }
I20260925 14:49:09.779733 8272 raft_consensus.cc:374] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:09.779793 8272 raft_consensus.cc:729] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e51e23e6933842148fc2d2847730cc4c, State: Initialized, Role: FOLLOWER
I20260925 14:49:09.779990 8272 consensus_queue.cc:260] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } }
I20260925 14:49:09.780057 8272 raft_consensus.cc:388] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:09.780114 8272 raft_consensus.cc:482] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:09.780169 8272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:09.780872 8272 raft_consensus.cc:504] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } }
I20260925 14:49:09.780963 8272 leader_election.cc:302] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e51e23e6933842148fc2d2847730cc4c; no voters:
I20260925 14:49:09.781104 8272 leader_election.cc:288] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:09.781140 8275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:09.781317 8275 raft_consensus.cc:686] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [term 1 LEADER]: Becoming Leader. State: Replica: e51e23e6933842148fc2d2847730cc4c, State: Running, Role: LEADER
I20260925 14:49:09.781406 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8204
I20260925 14:49:09.781483 8275 consensus_queue.cc:237] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } }
I20260925 14:49:09.781527 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:09.781699 22327 external_mini_cluster.cc:1428] Setting key 0e0fb66e941b69ecf3ff8c9f23b5be33
I20260925 14:49:09.781913 8277 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e51e23e6933842148fc2d2847730cc4c. Latest consensus state: current_term: 1 leader_uuid: "e51e23e6933842148fc2d2847730cc4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } } }
I20260925 14:49:09.782004 8277 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:09.782127 8272 sys_catalog.cc:564] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:09.782491 8276 sys_catalog.cc:455] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e51e23e6933842148fc2d2847730cc4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e51e23e6933842148fc2d2847730cc4c" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 45675 } } }
I20260925 14:49:09.782573 8276 sys_catalog.cc:458] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c [sys.catalog]: This master's current role is: LEADER
W20260925 14:49:09.783964 8294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:09.784081 8294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:09.784185 8280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:09.784372 8280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:09.786060 8280 catalog_manager.cc:1347] Generated new cluster ID: b077477e4dc947d18dad0036a25f5f6c
I20260925 14:49:09.786120 8280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:49:09 dist-test-slave-d7lw krb5kdc[8191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:09.795069 8280 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:09.795264 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:09.782565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60924 (local address 127.21.205.254:45675)
0925 14:49:09.782761 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:09.782772 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:09.783662 (+ 890us) server_negotiation.cc:407] Connection header received
0925 14:49:09.783837 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:09.783846 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:09.783911 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:09.784074 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:09.784866 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.786190 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.787290 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.787980 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.788081 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:09.791104 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:09.791109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:09.791109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:09.791123 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:09.793320 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:09.793759 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:09.793763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:09.793763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:09.793890 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:09.794130 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:09.794131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:09.794132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:09.794416 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:09.794571 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:09.794868 (+ 297us) server_negotiation.cc:299] Negotiation successful
0925 14:49:09.795026 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260925 14:49:09.796130 8280 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:09.800081 8280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: Generated new TSK 0
I20260925 14:49:09.800307 8280 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:49:09.826505 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:09.912660 8299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:09.912907 8299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:09.912950 8299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:09.914323 8299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:09.914391 8299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:09.914439 8299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:09.916049 8299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:09.916317 8299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:09.916546 8299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:09.922457 8305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:09.922468 8304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:09.922598 8307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:09.922823 8299 server_base.cc:1034] running on GCE node
I20260925 14:49:09.923046 8299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:09.923323 8299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:09.924475 8299 hybrid_clock.cc:648] HybridClock initialized: now 1790347749924463 us; error 28 us; skew 500 ppm
Sep 25 14:49:09 dist-test-slave-d7lw krb5kdc[8191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:09.926911 8299 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:49:09.927402 8299 webserver.cc:460] Webserver started at http://127.21.205.193:37075/ using document root <none> and password file <none>
I20260925 14:49:09.927654 8299 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:09.927700 8299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:09.927894 8299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:09.928972 8299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "49b3b9fe5c9e425594eebe473410c36c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "87350e35355a5ac6a6d81828e4cb44ed"
server_key_iv: "239e90ae5fd1e5b09a224fb2a30f5521"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.929342 8299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "49b3b9fe5c9e425594eebe473410c36c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "87350e35355a5ac6a6d81828e4cb44ed"
server_key_iv: "239e90ae5fd1e5b09a224fb2a30f5521"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.930706 8299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:09.931429 8314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:09.931638 8299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:09.931742 8299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "49b3b9fe5c9e425594eebe473410c36c"
format_stamp: "Formatted at 2026-09-25 14:49:09 on dist-test-slave-d7lw"
server_key: "87350e35355a5ac6a6d81828e4cb44ed"
server_key_iv: "239e90ae5fd1e5b09a224fb2a30f5521"
server_key_version: "encryptionkey@0"
I20260925 14:49:09.931847 8299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:09.965054 8299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:09.966990 8299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:09.967163 8299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:09.967408 8299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:09.967741 8299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:09.967833 8299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:09.967916 8299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:09.967948 8299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:09.973800 8299 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:43193
I20260925 14:49:09.973846 8427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:43193 every 8 connection(s)
I20260925 14:49:09.974171 8299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
Sep 25 14:49:09 dist-test-slave-d7lw krb5kdc[8191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:09.982411 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8299
I20260925 14:49:09.982527 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:09.982781 22327 external_mini_cluster.cc:1428] Setting key ad1f241f1f7070ec8cf23202cee16ec7
I20260925 14:49:09.983456 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:09.974843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:54113 (local address 127.21.205.254:45675)
0925 14:49:09.974926 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:09.974929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:09.975139 (+ 210us) server_negotiation.cc:407] Connection header received
0925 14:49:09.975632 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:09.975633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:09.975643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:09.975694 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:09.976666 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.977161 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.978102 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.978294 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.978343 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:09.980474 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:09.980476 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:09.980476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:09.980484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:09.981903 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:09.982375 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:09.982377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:09.982377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:09.982435 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:09.982769 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:09.982769 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:09.982770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:09.982912 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:09.982981 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:09.983282 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 14:49:09.983372 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 14:49:09.983666 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:09.974817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:45675 (local address 127.21.205.193:54113)
0925 14:49:09.975001 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:09.975039 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:09.975549 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:09.975749 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:09.975752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:09.976389 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:09.976585 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:09.976590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.977241 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.977242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:09.978028 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:09.978029 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:09.978372 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:09.978373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:09.978474 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:09.978950 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:09.978958 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:09.980375 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:09.982003 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:09.982010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:09.982012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:09.982297 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:09.982530 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:09.982531 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:09.982531 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:09.982694 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:09.982974 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:09.982981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:09.983112 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 14:49:09.983238 (+ 126us) client_negotiation.cc:240] Negotiation successful
0925 14:49:09.983393 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":104,"threads_started":1}
I20260925 14:49:09.984139 8428 heartbeater.cc:347] Connected to a master server at 127.21.205.254:45675
I20260925 14:49:09.984248 8428 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:09.984423 8428 heartbeater.cc:511] Master 127.21.205.254:45675 requested a full tablet report, sending...
I20260925 14:49:09.984972 8236 ts_manager.cc:194] Registered new tserver with Master: 49b3b9fe5c9e425594eebe473410c36c (127.21.205.193:43193)
I20260925 14:49:09.985589 8236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:54113
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:49:10.010965 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:10.091579 8433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.091831 8433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.091876 8433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.093046 8433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.093114 8433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.093165 8433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:10.095186 8433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.095458 8433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.095728 8433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.100535 8439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.100608 8433 server_base.cc:1034] running on GCE node
W20260925 14:49:10.100646 8438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.100536 8441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.100904 8433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.101152 8433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.102300 8433 hybrid_clock.cc:648] HybridClock initialized: now 1790347750102287 us; error 28 us; skew 500 ppm
Sep 25 14:49:10 dist-test-slave-d7lw krb5kdc[8191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:10.105268 8433 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:49:10.105696 8433 webserver.cc:460] Webserver started at http://127.21.205.194:38183/ using document root <none> and password file <none>
I20260925 14:49:10.105989 8433 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.106062 8433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.106217 8433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.107615 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "ac1cb250e136435cac25c052c74d1013"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "5d24ef93d9320d38fe5a3d09e46c54cc"
server_key_iv: "3ae1f602e4dde9875f55952ce8942e25"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.108032 8433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "ac1cb250e136435cac25c052c74d1013"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "5d24ef93d9320d38fe5a3d09e46c54cc"
server_key_iv: "3ae1f602e4dde9875f55952ce8942e25"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.109254 8433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:10.109894 8448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.110062 8433 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:10.110153 8433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "ac1cb250e136435cac25c052c74d1013"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "5d24ef93d9320d38fe5a3d09e46c54cc"
server_key_iv: "3ae1f602e4dde9875f55952ce8942e25"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.110239 8433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.128480 8433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.130822 8433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.130990 8433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.131225 8433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:10.131525 8433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:10.131577 8433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.131623 8433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:10.131651 8433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.137770 8433 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:36697
I20260925 14:49:10.137866 8563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:36697 every 8 connection(s)
I20260925 14:49:10.138175 8433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
Sep 25 14:49:10 dist-test-slave-d7lw krb5kdc[8191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:10.145491 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8433
I20260925 14:49:10.145607 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:10.145862 22327 external_mini_cluster.cc:1428] Setting key 770ec5b9f3182712d4701723ce467ee6
I20260925 14:49:10.147013 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.138922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:59447 (local address 127.21.205.254:45675)
0925 14:49:10.139006 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.139009 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.139192 (+ 183us) server_negotiation.cc:407] Connection header received
0925 14:49:10.139682 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.139683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.139693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.139763 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.140676 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.141184 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.142096 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.142271 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.142319 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.144357 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.144359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.144359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:10.144365 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.145519 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.146044 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.146045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.146045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.146110 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.146374 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.146375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.146375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.146494 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.146560 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.146882 (+ 322us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.146957 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 14:49:10.147321 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.138894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:45675 (local address 127.21.205.194:59447)
0925 14:49:10.139074 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.139110 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.139598 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.139872 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.139876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.140410 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:10.140602 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.140606 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.141256 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.141256 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.142025 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.142026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.142339 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.142340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.142438 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.142970 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:10.142983 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:10.144273 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:10.145647 (+ 1374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:10.145654 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:10.145656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:10.145957 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:10.146176 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:10.146177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:10.146177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:10.146313 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:10.146562 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:10.146567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:10.146719 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.146865 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.147028 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260925 14:49:10.147825 8564 heartbeater.cc:347] Connected to a master server at 127.21.205.254:45675
I20260925 14:49:10.147933 8564 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:10.148129 8564 heartbeater.cc:511] Master 127.21.205.254:45675 requested a full tablet report, sending...
I20260925 14:49:10.148538 8236 ts_manager.cc:194] Registered new tserver with Master: ac1cb250e136435cac25c052c74d1013 (127.21.205.194:36697)
I20260925 14:49:10.149051 8236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:59447
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:49:10.168354 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:10.246992 8569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.247182 8569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.247205 8569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.248332 8569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.248369 8569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.248394 8569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:10.249814 8569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:45675
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.250036 8569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.250272 8569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.255213 8577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.255285 8569 server_base.cc:1034] running on GCE node
W20260925 14:49:10.255226 8575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.255373 8574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.255553 8569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.255739 8569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.256919 8569 hybrid_clock.cc:648] HybridClock initialized: now 1790347750256908 us; error 27 us; skew 500 ppm
Sep 25 14:49:10 dist-test-slave-d7lw krb5kdc[8191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:10.259174 8569 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:49:10.259619 8569 webserver.cc:460] Webserver started at http://127.21.205.195:46211/ using document root <none> and password file <none>
I20260925 14:49:10.259943 8569 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.260015 8569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.260149 8569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.261183 8569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "57c24850f1e34f6cb787e6acd70f455e"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b36473ebc904f826c4e5fec1ea46f989"
server_key_iv: "66cd5065987584f25c6cdccbccb1de67"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.261547 8569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "57c24850f1e34f6cb787e6acd70f455e"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b36473ebc904f826c4e5fec1ea46f989"
server_key_iv: "66cd5065987584f25c6cdccbccb1de67"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.262828 8569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:10.263620 8584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.263850 8569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:10.263953 8569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "57c24850f1e34f6cb787e6acd70f455e"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b36473ebc904f826c4e5fec1ea46f989"
server_key_iv: "66cd5065987584f25c6cdccbccb1de67"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.264040 8569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.289609 8569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.291366 8569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.291524 8569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.291745 8569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:10.292062 8569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:10.292115 8569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.292155 8569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:10.292183 8569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.298200 8569 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:46349
I20260925 14:49:10.298246 8699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:46349 every 8 connection(s)
I20260925 14:49:10.298539 8569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:49:10.302546 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8569
I20260925 14:49:10.302640 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:10.302804 22327 external_mini_cluster.cc:1428] Setting key 994e59c1e32ed20ceecfd4ebc06cd3a3
Sep 25 14:49:10 dist-test-slave-d7lw krb5kdc[8191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347750, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:10.307631 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.299184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:34257 (local address 127.21.205.254:45675)
0925 14:49:10.299269 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.299272 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.299437 (+ 165us) server_negotiation.cc:407] Connection header received
0925 14:49:10.300007 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.300008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.300018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.300074 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.300938 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.301408 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.302317 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.302574 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.302649 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.304901 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.304902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.304903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:10.304909 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.306454 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.306866 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.306866 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.306867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.306914 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.307135 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.307136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.307136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.307231 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.307280 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.307509 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.307566 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 14:49:10.307857 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.299161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:45675 (local address 127.21.205.195:34257)
0925 14:49:10.299327 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.299360 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.299905 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.300146 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.300151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.300681 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:10.300859 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.300863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.301484 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.301484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.302249 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.302251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.302666 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.302667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.302789 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.303350 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:10.303359 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:10.304804 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:10.306529 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:10.306552 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:10.306553 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:10.306793 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:10.306974 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:10.306975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:10.306975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:10.307077 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:10.307289 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:10.307293 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:10.307400 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.307485 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.307620 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":96,"threads_started":1}
I20260925 14:49:10.308346 8700 heartbeater.cc:347] Connected to a master server at 127.21.205.254:45675
I20260925 14:49:10.308427 8700 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:10.308617 8700 heartbeater.cc:511] Master 127.21.205.254:45675 requested a full tablet report, sending...
I20260925 14:49:10.308992 8236 ts_manager.cc:194] Registered new tserver with Master: 57c24850f1e34f6cb787e6acd70f455e (127.21.205.195:46349)
I20260925 14:49:10.309338 8236 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:34257
I20260925 14:49:10.314484 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:10.321488 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.315828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.21.205.254:45675)
0925 14:49:10.315878 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.315879 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.315983 (+ 104us) server_negotiation.cc:407] Connection header received
0925 14:49:10.316019 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.316020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.316029 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.316091 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.316573 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.317033 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.317702 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.317873 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.317924 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.319114 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.319116 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.319116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:10.319122 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.320381 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.320748 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.320749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.320749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.320803 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:10.321031 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:10.321032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:10.321032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:10.321171 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.321267 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.321380 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.321428 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:10.323058 8236 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:10.324687 8236 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:10.331241 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.327021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:36697 (local address 127.0.0.1:36942)
0925 14:49:10.327641 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.327651 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.327701 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.327926 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.327927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.327929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:10.327967 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.327969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.329243 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.329244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.329971 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.329973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.330942 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.331027 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.331043 (+ 16us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.331108 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.331163 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":71,"threads_started":1}
I20260925 14:49:10.331455 8711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.326821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43193 (local address 127.0.0.1:41682)
0925 14:49:10.326921 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.326958 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.327036 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.327546 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.327548 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.327551 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:10.327959 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.327961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.328877 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.328878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.330340 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.330342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.331088 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.331089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.331137 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.331140 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.331190 (+ 50us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.331221 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":8,"thread_start_us":59,"threads_started":1}
I20260925 14:49:10.331445 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.327165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.21.205.195:46349)
0925 14:49:10.327335 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.327340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.327406 (+ 66us) server_negotiation.cc:407] Connection header received
0925 14:49:10.327486 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.327489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.327506 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.327589 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:10.327705 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.328922 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330191 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.330897 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330956 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.330999 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.331299 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.331346 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260925 14:49:10.331598 8716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.327165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.21.205.194:36697)
0925 14:49:10.327335 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.327340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.327763 (+ 423us) server_negotiation.cc:407] Connection header received
0925 14:49:10.327825 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.327827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.327843 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.327918 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:10.328056 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.329145 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330058 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.330801 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330934 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.330991 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.331452 (+ 461us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.331497 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
I20260925 14:49:10.331625 8712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.326825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:46349 (local address 127.0.0.1:49960)
0925 14:49:10.327354 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.327363 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.327419 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.327585 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.327585 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.327587 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:10.327650 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.327652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.329021 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.329022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.330108 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.330110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.331033 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.331034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.331121 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.331125 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.331179 (+ 54us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.331220 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":1,"thread_start_us":74,"threads_started":1}
I20260925 14:49:10.332311 8496 tablet_service.cc:1467] Processing CreateTablet for tablet e1572fbb811a437989620a475ddd0cd5 (DEFAULT_TABLE table=test-table [id=739eed6a7b50412d853fb3060cb64068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:10.332314 8633 tablet_service.cc:1467] Processing CreateTablet for tablet e1572fbb811a437989620a475ddd0cd5 (DEFAULT_TABLE table=test-table [id=739eed6a7b50412d853fb3060cb64068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:10.332661 8496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1572fbb811a437989620a475ddd0cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:10.332669 8633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1572fbb811a437989620a475ddd0cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:10.335363 8718 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Bootstrap starting.
I20260925 14:49:10.331488 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.326980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.21.205.193:43193)
0925 14:49:10.327382 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.327386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.327395 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:10.327437 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.327438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.327451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.327540 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:10.328024 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.328758 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.330415 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.331010 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.331098 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.331154 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.331209 (+ 55us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.331247 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":108,"threads_started":1}
I20260925 14:49:10.335939 8717 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Bootstrap starting.
I20260925 14:49:10.336066 8362 tablet_service.cc:1467] Processing CreateTablet for tablet e1572fbb811a437989620a475ddd0cd5 (DEFAULT_TABLE table=test-table [id=739eed6a7b50412d853fb3060cb64068]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:10.336176 8718 tablet_bootstrap.cc:654] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:10.336380 8362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1572fbb811a437989620a475ddd0cd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:10.336602 8717 tablet_bootstrap.cc:654] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:10.336786 8718 log.cc:824] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:10.337198 8717 log.cc:824] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:10.337597 8718 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: No bootstrap required, opened a new log
I20260925 14:49:10.337715 8718 ts_tablet_manager.cc:1397] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 14:49:10.337975 8717 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: No bootstrap required, opened a new log
I20260925 14:49:10.338052 8717 ts_tablet_manager.cc:1397] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:10.339206 8721 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: Bootstrap starting.
I20260925 14:49:10.339774 8721 tablet_bootstrap.cc:654] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:10.340006 8717 raft_consensus.cc:348] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.340034 8718 raft_consensus.cc:348] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.340214 8717 raft_consensus.cc:374] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:10.340214 8718 raft_consensus.cc:374] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:10.340251 8718 raft_consensus.cc:729] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1cb250e136435cac25c052c74d1013, State: Initialized, Role: FOLLOWER
I20260925 14:49:10.340251 8717 raft_consensus.cc:729] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57c24850f1e34f6cb787e6acd70f455e, State: Initialized, Role: FOLLOWER
I20260925 14:49:10.340335 8721 log.cc:824] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:10.340386 8717 consensus_queue.cc:260] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.340386 8718 consensus_queue.cc:260] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.340687 8717 ts_tablet_manager.cc:1428] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 14:49:10.340687 8718 ts_tablet_manager.cc:1428] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 14:49:10.340807 8564 heartbeater.cc:503] Master 127.21.205.254:45675 was elected leader, sending a full tablet report...
I20260925 14:49:10.340831 8700 heartbeater.cc:503] Master 127.21.205.254:45675 was elected leader, sending a full tablet report...
I20260925 14:49:10.341959 8721 tablet_bootstrap.cc:492] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: No bootstrap required, opened a new log
I20260925 14:49:10.342033 8721 ts_tablet_manager.cc:1397] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260925 14:49:10.343269 8721 raft_consensus.cc:348] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.343413 8721 raft_consensus.cc:374] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:10.343464 8721 raft_consensus.cc:729] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49b3b9fe5c9e425594eebe473410c36c, State: Initialized, Role: FOLLOWER
I20260925 14:49:10.343592 8721 consensus_queue.cc:260] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.343891 8721 ts_tablet_manager.cc:1428] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:10.343945 8428 heartbeater.cc:503] Master 127.21.205.254:45675 was elected leader, sending a full tablet report...
I20260925 14:49:10.377348 8725 raft_consensus.cc:482] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:10.377509 8725 raft_consensus.cc:504] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.377892 8725 leader_election.cc:288] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57c24850f1e34f6cb787e6acd70f455e (127.21.205.195:46349), ac1cb250e136435cac25c052c74d1013 (127.21.205.194:36697)
I20260925 14:49:10.380766 8715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.378140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:58633 (local address 127.21.205.195:46349)
0925 14:49:10.378201 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.378203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.378214 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:10.378244 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.378245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.378255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.378342 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:10.378507 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.378995 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.379990 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.380370 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.380477 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.380486 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.380665 (+ 179us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.380702 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:49:10.380766 8430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.378059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:46349 (local address 127.21.205.193:58633)
0925 14:49:10.378112 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.378124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.378168 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.378348 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.378349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.378351 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:10.378430 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.378432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.379103 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.379105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.379904 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.379906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.380477 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.380478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.380562 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.380564 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.380646 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.380692 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:49:10.380942 8716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.378304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:36319 (local address 127.21.205.194:36697)
0925 14:49:10.378353 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.378354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.378402 (+ 48us) server_negotiation.cc:407] Connection header received
0925 14:49:10.378466 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.378467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.378478 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.378549 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:10.378714 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.379190 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.380176 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.380565 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.380672 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.380680 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.380831 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.380870 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:10.380952 8726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.378193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:36697 (local address 127.21.205.193:36319)
0925 14:49:10.378353 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.378363 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.378415 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.378555 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.378556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.378557 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:10.378633 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.378635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.379287 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.379288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.380086 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.380088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.380671 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.380672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.380748 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.380756 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.380824 (+ 68us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.380863 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":103,"threads_started":1}
I20260925 14:49:10.381181 8653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1572fbb811a437989620a475ddd0cd5" candidate_uuid: "49b3b9fe5c9e425594eebe473410c36c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57c24850f1e34f6cb787e6acd70f455e" is_pre_election: true
I20260925 14:49:10.381345 8653 raft_consensus.cc:2454] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b3b9fe5c9e425594eebe473410c36c in term 0.
I20260925 14:49:10.381575 8316 leader_election.cc:302] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b3b9fe5c9e425594eebe473410c36c, 57c24850f1e34f6cb787e6acd70f455e; no voters:
I20260925 14:49:10.381556 8517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1572fbb811a437989620a475ddd0cd5" candidate_uuid: "49b3b9fe5c9e425594eebe473410c36c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1cb250e136435cac25c052c74d1013" is_pre_election: true
I20260925 14:49:10.381688 8517 raft_consensus.cc:2454] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 49b3b9fe5c9e425594eebe473410c36c in term 0.
I20260925 14:49:10.381706 8725 raft_consensus.cc:2781] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:10.381770 8725 raft_consensus.cc:482] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:10.381796 8725 raft_consensus.cc:3037] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:10.382706 8725 raft_consensus.cc:504] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.382836 8725 leader_election.cc:288] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [CANDIDATE]: Term 1 election: Requested vote from peers 57c24850f1e34f6cb787e6acd70f455e (127.21.205.195:46349), ac1cb250e136435cac25c052c74d1013 (127.21.205.194:36697)
I20260925 14:49:10.383045 8517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1572fbb811a437989620a475ddd0cd5" candidate_uuid: "49b3b9fe5c9e425594eebe473410c36c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac1cb250e136435cac25c052c74d1013"
I20260925 14:49:10.383031 8653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1572fbb811a437989620a475ddd0cd5" candidate_uuid: "49b3b9fe5c9e425594eebe473410c36c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57c24850f1e34f6cb787e6acd70f455e"
I20260925 14:49:10.383131 8517 raft_consensus.cc:3037] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:10.383116 8653 raft_consensus.cc:3037] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:10.384060 8653 raft_consensus.cc:2454] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b3b9fe5c9e425594eebe473410c36c in term 1.
I20260925 14:49:10.384060 8517 raft_consensus.cc:2454] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49b3b9fe5c9e425594eebe473410c36c in term 1.
I20260925 14:49:10.384203 8316 leader_election.cc:302] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49b3b9fe5c9e425594eebe473410c36c, 57c24850f1e34f6cb787e6acd70f455e; no voters:
I20260925 14:49:10.384330 8725 raft_consensus.cc:2781] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:10.384491 8725 raft_consensus.cc:686] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [term 1 LEADER]: Becoming Leader. State: Replica: 49b3b9fe5c9e425594eebe473410c36c, State: Running, Role: LEADER
I20260925 14:49:10.384591 8725 consensus_queue.cc:237] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } }
I20260925 14:49:10.385381 8235 catalog_manager.cc:5495] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c reported cstate change: term changed from 0 to 1, leader changed from <none> to 49b3b9fe5c9e425594eebe473410c36c (127.21.205.193). New cstate: current_term: 1 leader_uuid: "49b3b9fe5c9e425594eebe473410c36c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49b3b9fe5c9e425594eebe473410c36c" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 43193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 } health_report { overall_health: UNKNOWN } } }
W20260925 14:49:10.388736 8565 tablet.cc:2369] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:49:10.403963 8713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.401519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.21.205.193:43193)
0925 14:49:10.401593 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.401595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.401606 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:10.401715 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.401716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.401743 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.401825 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:10.401959 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.402426 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.403158 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.403361 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.403444 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.403596 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:10.403689 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:10.403828 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.403859 (+ 31us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.403871 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 14:49:10.405879 8517 raft_consensus.cc:1261] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 1 FOLLOWER]: Refusing update from remote peer 49b3b9fe5c9e425594eebe473410c36c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:10.405905 8653 raft_consensus.cc:1261] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 1 FOLLOWER]: Refusing update from remote peer 49b3b9fe5c9e425594eebe473410c36c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:10.406212 8725 consensus_queue.cc:1035] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac1cb250e136435cac25c052c74d1013" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 36697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:10.406350 8727 consensus_queue.cc:1035] T e1572fbb811a437989620a475ddd0cd5 P 49b3b9fe5c9e425594eebe473410c36c [LEADER]: Connected to new peer: Peer: permanent_uuid: "57c24850f1e34f6cb787e6acd70f455e" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:10.408955 8730 mvcc.cc:205] Tried to move back new op lower bound from 7333264385659310080 to 7333264385575813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333264385659310080}]
I20260925 14:49:10.409359 8731 mvcc.cc:205] Tried to move back new op lower bound from 7333264385659310080 to 7333264385575813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333264385659310080}]
I20260925 14:49:10.409765 8732 mvcc.cc:205] Tried to move back new op lower bound from 7333264385659310080 to 7333264385575813120. Current Snapshot: MvccSnapshot[applied={T|T < 7333264385659310080 or (T in {7333264385659310080})}]
I20260925 14:49:10.411723 8235 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:10.411852 8235 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:10.412586 8235 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e51e23e6933842148fc2d2847730cc4c: Sending DeleteTablet for 3 replicas of tablet e1572fbb811a437989620a475ddd0cd5
I20260925 14:49:10.412909 8633 tablet_service.cc:1514] Processing DeleteTablet for tablet e1572fbb811a437989620a475ddd0cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:10 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:49960
I20260925 14:49:10.413185 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8299
I20260925 14:49:10.413169 8496 tablet_service.cc:1514] Processing DeleteTablet for tablet e1572fbb811a437989620a475ddd0cd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:10 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:36942
I20260925 14:49:10.414004 8740 tablet_replica.cc:330] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: stopping tablet replica
I20260925 14:49:10.414101 8740 raft_consensus.cc:2229] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:10.414273 8740 raft_consensus.cc:2258] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:10.414500 8739 tablet_replica.cc:330] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: stopping tablet replica
I20260925 14:49:10.414593 8739 raft_consensus.cc:2229] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:10.414690 8739 raft_consensus.cc:2258] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:10.414739 8740 ts_tablet_manager.cc:1905] T e1572fbb811a437989620a475ddd0cd5 P ac1cb250e136435cac25c052c74d1013: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:10.415108 8739 ts_tablet_manager.cc:1905] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:10.416785 8739 ts_tablet_manager.cc:1918] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:10.416844 8739 log.cc:1196] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/e1572fbb811a437989620a475ddd0cd5
I20260925 14:49:10.417109 8739 ts_tablet_manager.cc:1939] T e1572fbb811a437989620a475ddd0cd5 P 57c24850f1e34f6cb787e6acd70f455e: Deleting consensus metadata
W20260925 14:49:10.419471 8220 connection.cc:535] client connection to 127.21.205.193:43193 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260925 14:49:10.419564 8220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 14:49:10.419893 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8433
I20260925 14:49:10.420050 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.419896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:43193 (local address 127.0.0.1:41694)
0925 14:49:10.419964 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.420010 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 14:49:10.420642 8220 catalog_manager.cc:4584] TS 49b3b9fe5c9e425594eebe473410c36c (127.21.205.193:43193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1572fbb811a437989620a475ddd0cd5: Network error: Client connection negotiation failed: client connection to 127.21.205.193:43193: connect: Connection refused (error 111)
I20260925 14:49:10.421634 8221 catalog_manager.cc:4841] TS 57c24850f1e34f6cb787e6acd70f455e (127.21.205.195:46349): tablet e1572fbb811a437989620a475ddd0cd5 (table test-table [id=739eed6a7b50412d853fb3060cb64068]) successfully deleted
I20260925 14:49:10.425351 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8569
I20260925 14:49:10.425382 8714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.425274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:36697 (local address 127.0.0.1:36944)
0925 14:49:10.425328 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.425359 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.194:36697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 14:49:10.430476 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8204
2026-09-25T14:49:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1986 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T14:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:10Z Disabled control of system clock
I20260925 14:49:10.457686 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32979
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:42121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:32979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:10.536942 8745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.537177 8745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.537197 8745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.538271 8745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:10.538305 8745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.538318 8745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.538331 8745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:10.538343 8745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:10.539641 8745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:42121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:32979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:32979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.539884 8745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.540143 8745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.544739 8751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.544797 8745 server_base.cc:1034] running on GCE node
W20260925 14:49:10.544713 8753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.544914 8750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.545266 8745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.545544 8745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.546705 8745 hybrid_clock.cc:648] HybridClock initialized: now 1790347750546691 us; error 24 us; skew 500 ppm
I20260925 14:49:10.547973 8745 webserver.cc:460] Webserver started at http://127.21.205.254:46041/ using document root <none> and password file <none>
I20260925 14:49:10.548225 8745 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.548295 8745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.548430 8745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.549479 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "9ea352d635e343b3ba9cacb8dfee3be1"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4aa49717d18e6cd89b6b603e6e70b330"
server_key_iv: "ed6710edf333b6ff36d26e1253ca758a"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.549842 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "9ea352d635e343b3ba9cacb8dfee3be1"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4aa49717d18e6cd89b6b603e6e70b330"
server_key_iv: "ed6710edf333b6ff36d26e1253ca758a"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.551204 8745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:10.551986 8759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.552161 8745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:10.552260 8745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "9ea352d635e343b3ba9cacb8dfee3be1"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4aa49717d18e6cd89b6b603e6e70b330"
server_key_iv: "ed6710edf333b6ff36d26e1253ca758a"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.552367 8745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.573477 8745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.573825 8745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.573983 8745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.578047 8745 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:32979
I20260925 14:49:10.578081 8811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:32979 every 8 connection(s)
I20260925 14:49:10.578485 8745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:10.579444 8812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:10.581785 8812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: Bootstrap starting.
I20260925 14:49:10.581935 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8745
I20260925 14:49:10.582037 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:10.582262 22327 external_mini_cluster.cc:1428] Setting key 608ebd3dfba446f2b1414a14445a991a
I20260925 14:49:10.582558 8812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:10.583017 8812 log.cc:824] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:10.583650 8812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: No bootstrap required, opened a new log
I20260925 14:49:10.585398 8812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } }
I20260925 14:49:10.585556 8812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:10.585623 8812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea352d635e343b3ba9cacb8dfee3be1, State: Initialized, Role: FOLLOWER
I20260925 14:49:10.585796 8812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } }
I20260925 14:49:10.585892 8812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:10.585937 8812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:10.585984 8812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:10.586499 8812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } }
I20260925 14:49:10.586622 8812 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea352d635e343b3ba9cacb8dfee3be1; no voters:
I20260925 14:49:10.586805 8812 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:10.586844 8817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:10.587002 8817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea352d635e343b3ba9cacb8dfee3be1, State: Running, Role: LEADER
I20260925 14:49:10.587087 8812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:10.587092 8817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } }
I20260925 14:49:10.587461 8818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } } }
I20260925 14:49:10.587440 8817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea352d635e343b3ba9cacb8dfee3be1. Latest consensus state: current_term: 1 leader_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea352d635e343b3ba9cacb8dfee3be1" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 32979 } } }
I20260925 14:49:10.587528 8817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:10.587759 8818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:10.587911 8823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:10.588073 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.582709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44286 (local address 127.21.205.254:32979)
0925 14:49:10.582935 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.582943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.582967 (+ 24us) server_negotiation.cc:407] Connection header received
0925 14:49:10.583429 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.583436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.583496 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.583650 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.583866 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.584754 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.585557 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.586167 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.586301 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.587039 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.587043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.587043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:10.587272 (+ 229us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.587314 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:10.587328 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.587403 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.587601 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.587694 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":77,"threads_started":1}
I20260925 14:49:10.588373 8823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:10.589921 8823 catalog_manager.cc:1347] Generated new cluster ID: 7ed2c44f06904518bb1375e1d2b9ad1b
I20260925 14:49:10.589970 8823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:10.599905 8823 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:10.600407 8823 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:10.604652 8823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: Generated new TSK 0
I20260925 14:49:10.604820 8823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:10.606348 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--builtin_ntp_servers=127.21.205.212:42121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:10.688748 8837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.688993 8837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.689016 8837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.690146 8837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.690184 8837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.690210 8837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:10.691615 8837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:42121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.691866 8837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.692186 8837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.697196 8842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.697320 8845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.697192 8843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.697463 8837 server_base.cc:1034] running on GCE node
I20260925 14:49:10.697793 8837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.698128 8837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.699313 8837 hybrid_clock.cc:648] HybridClock initialized: now 1790347750699300 us; error 43 us; skew 500 ppm
I20260925 14:49:10.700837 8837 webserver.cc:460] Webserver started at http://127.21.205.193:39747/ using document root <none> and password file <none>
I20260925 14:49:10.701143 8837 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.701206 8837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.701344 8837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.702786 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "0e635926400d4995849c35db50ebf585"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "f30c955d495e7d34704450f2b73aed4f"
server_key_iv: "7837006ce120935648ff13c50c976856"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.703228 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "0e635926400d4995849c35db50ebf585"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "f30c955d495e7d34704450f2b73aed4f"
server_key_iv: "7837006ce120935648ff13c50c976856"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.704983 8837 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260925 14:49:10.705932 8851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.706139 8837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:10.706238 8837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "0e635926400d4995849c35db50ebf585"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "f30c955d495e7d34704450f2b73aed4f"
server_key_iv: "7837006ce120935648ff13c50c976856"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.706315 8837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.738356 8837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.738620 8837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.738729 8837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.738915 8837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:10.739192 8837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:10.739224 8837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.739249 8837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:10.739269 8837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.746496 8837 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:40405
I20260925 14:49:10.746520 8964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:40405 every 8 connection(s)
I20260925 14:49:10.746905 8837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:49:10.751596 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8837
I20260925 14:49:10.751721 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:10.751950 22327 external_mini_cluster.cc:1428] Setting key d926bf776374571e5a6e7ad89d10c765
I20260925 14:49:10.752059 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.747678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:50179 (local address 127.21.205.254:32979)
0925 14:49:10.747724 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.747727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.748012 (+ 285us) server_negotiation.cc:407] Connection header received
0925 14:49:10.748518 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.748519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.748531 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.748587 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.749210 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.749711 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.750656 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.750837 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.750898 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.751760 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.751761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.751762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:10.751766 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.751799 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:10.751803 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.751853 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.751958 (+ 105us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.751982 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:10.752312 8967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.747684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32979 (local address 127.21.205.193:50179)
0925 14:49:10.747882 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.747914 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.748401 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.748648 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.748651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.748659 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:10.749140 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.749144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.749791 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.749791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.750580 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.750582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.750918 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.750918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.751017 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.751597 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:10.751598 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:10.751624 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:10.751645 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:10.751853 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:10.751860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:10.751864 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.751946 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.752045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260925 14:49:10.752772 8965 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32979
I20260925 14:49:10.752873 8965 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:10.753027 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--builtin_ntp_servers=127.21.205.212:42121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 14:49:10.753074 8965 heartbeater.cc:511] Master 127.21.205.254:32979 requested a full tablet report, sending...
I20260925 14:49:10.753600 8775 ts_manager.cc:194] Registered new tserver with Master: 0e635926400d4995849c35db50ebf585 (127.21.205.193:40405)
I20260925 14:49:10.754485 8775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.193:50179
W20260925 14:49:10.833750 8968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.833935 8968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.833956 8968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.835011 8968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.835041 8968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.835070 8968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:10.836333 8968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:42121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.836508 8968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.836737 8968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.841651 8974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.841703 8975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.841645 8977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.841929 8968 server_base.cc:1034] running on GCE node
I20260925 14:49:10.842165 8968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.842411 8968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.843583 8968 hybrid_clock.cc:648] HybridClock initialized: now 1790347750843567 us; error 24 us; skew 500 ppm
I20260925 14:49:10.844913 8968 webserver.cc:460] Webserver started at http://127.21.205.194:32897/ using document root <none> and password file <none>
I20260925 14:49:10.845180 8968 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.845249 8968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.845395 8968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.846441 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "dd05621120fb4f29b94ab0f4627ffed2"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b49fd1928000ff2766d1805eac645088"
server_key_iv: "fac75e5896f61b99ec1fbef754bd5ca2"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.846799 8968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "dd05621120fb4f29b94ab0f4627ffed2"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b49fd1928000ff2766d1805eac645088"
server_key_iv: "fac75e5896f61b99ec1fbef754bd5ca2"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.848150 8968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:10.848881 8983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.849050 8968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:10.849152 8968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "dd05621120fb4f29b94ab0f4627ffed2"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "b49fd1928000ff2766d1805eac645088"
server_key_iv: "fac75e5896f61b99ec1fbef754bd5ca2"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.849237 8968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.862705 8968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.862969 8968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.863127 8968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.863341 8968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:10.863656 8968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:10.863710 8968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.863749 8968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:10.863828 8968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.869333 8968 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:38335
I20260925 14:49:10.869374 9096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:38335 every 8 connection(s)
I20260925 14:49:10.869690 8968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:49:10.874615 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.870438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:54471 (local address 127.21.205.254:32979)
0925 14:49:10.870503 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.870505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.870705 (+ 200us) server_negotiation.cc:407] Connection header received
0925 14:49:10.871143 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.871143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.871153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.871208 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.871903 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.872412 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.873371 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.873564 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.873625 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.874400 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:10.874401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:10.874402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:10.874405 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:10.874415 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:10.874419 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:10.874458 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:10.874541 (+ 83us) server_negotiation.cc:299] Negotiation successful
0925 14:49:10.874560 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:49:10.874866 9099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.870390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32979 (local address 127.21.205.194:54471)
0925 14:49:10.870571 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.870615 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.871059 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.871261 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.871264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.871272 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:10.871746 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.871750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.872486 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.872487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.873265 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.873267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.873644 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.873644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.873748 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:10.874278 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:10.874278 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:10.874305 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:10.874326 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:10.874472 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:10.874476 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:10.874478 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:10.874529 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:49:10.874610 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":83,"threads_started":1}
I20260925 14:49:10.875355 9097 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32979
I20260925 14:49:10.875450 9097 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:10.875644 9097 heartbeater.cc:511] Master 127.21.205.254:32979 requested a full tablet report, sending...
I20260925 14:49:10.876029 8775 ts_manager.cc:194] Registered new tserver with Master: dd05621120fb4f29b94ab0f4627ffed2 (127.21.205.194:38335)
I20260925 14:49:10.876395 8775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.194:54471
I20260925 14:49:10.878582 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 8968
I20260925 14:49:10.878710 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:10.878902 22327 external_mini_cluster.cc:1428] Setting key 9eb5fbb8aa2ad50d4cfbaa74864e7aa2
I20260925 14:49:10.879909 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--builtin_ntp_servers=127.21.205.212:42121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:10.961208 9100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:10.961400 9100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:10.961418 9100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:10.962527 9100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:10.962565 9100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:10.962589 9100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:10.964038 9100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:42121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:32979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9100
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:10.964273 9100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:10.964552 9100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:10.969672 9105 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.969689 9108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:10.969704 9106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:10.969967 9100 server_base.cc:1034] running on GCE node
I20260925 14:49:10.970202 9100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:10.970403 9100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:10.971547 9100 hybrid_clock.cc:648] HybridClock initialized: now 1790347750971535 us; error 27 us; skew 500 ppm
I20260925 14:49:10.972877 9100 webserver.cc:460] Webserver started at http://127.21.205.195:45209/ using document root <none> and password file <none>
I20260925 14:49:10.973061 9100 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:10.973101 9100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:10.973192 9100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:10.974256 9100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "561df3840174493c941d94173fab2984"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4c3ab40d02226bd6d3d7ef539eb09446"
server_key_iv: "bcd948804898d375c690ab43f53af011"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.974553 9100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "561df3840174493c941d94173fab2984"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4c3ab40d02226bd6d3d7ef539eb09446"
server_key_iv: "bcd948804898d375c690ab43f53af011"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.975837 9100 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:10.976473 9115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.976655 9100 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:10.976749 9100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "561df3840174493c941d94173fab2984"
format_stamp: "Formatted at 2026-09-25 14:49:10 on dist-test-slave-d7lw"
server_key: "4c3ab40d02226bd6d3d7ef539eb09446"
server_key_iv: "bcd948804898d375c690ab43f53af011"
server_key_version: "encryptionkey@0"
I20260925 14:49:10.976827 9100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:10.989245 9100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:10.989514 9100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:10.989655 9100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:10.989862 9100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:10.990154 9100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:10.990223 9100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.990269 9100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:10.990295 9100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:10.995813 9100 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:46043
I20260925 14:49:10.995885 9228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:46043 every 8 connection(s)
I20260925 14:49:10.996227 9100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:49:11.000865 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.996925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:39521 (local address 127.21.205.254:32979)
0925 14:49:10.996989 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:10.996992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:10.997186 (+ 194us) server_negotiation.cc:407] Connection header received
0925 14:49:10.997623 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:10.997624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:10.997633 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:10.997700 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:10.998349 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.998813 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.999702 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.999940 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.999991 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.000637 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:11.000638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:11.000638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:11.000642 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:11.000673 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:11.000677 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:11.000722 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.000790 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.000809 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:49:11.001029 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:10.996901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:32979 (local address 127.21.205.195:39521)
0925 14:49:10.997067 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:10.997104 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:10.997535 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:10.997754 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:10.997757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:10.997764 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:10.998283 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.998287 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:10.998885 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:10.998886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:10.999632 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:10.999633 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.000006 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.000007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.000108 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.000523 (+ 415us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:11.000523 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:11.000547 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:11.000566 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:11.000724 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:11.000726 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:11.000728 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.000779 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.000848 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260925 14:49:11.001425 9229 heartbeater.cc:347] Connected to a master server at 127.21.205.254:32979
I20260925 14:49:11.001509 9229 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:11.001643 9229 heartbeater.cc:511] Master 127.21.205.254:32979 requested a full tablet report, sending...
I20260925 14:49:11.001986 8775 ts_manager.cc:194] Registered new tserver with Master: 561df3840174493c941d94173fab2984 (127.21.205.195:46043)
I20260925 14:49:11.002300 8775 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.195:39521
I20260925 14:49:11.004561 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9100
I20260925 14:49:11.004669 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:11.004874 22327 external_mini_cluster.cc:1428] Setting key 66109e27280841fcf9fdc579b49abe6c
I20260925 14:49:11.005671 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:11.009662 8815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.006954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.21.205.254:32979)
0925 14:49:11.007006 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.007007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.007132 (+ 125us) server_negotiation.cc:407] Connection header received
0925 14:49:11.007178 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.007178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.007186 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.007229 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:11.007391 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.007974 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.008640 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.008805 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.008849 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.009501 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:11.009502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:11.009502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:11.009505 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:11.009513 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:11.009516 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:11.009569 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.009609 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.009617 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:49:11.011346 8775 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:11.012916 8775 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:11.018795 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.015110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:46043 (local address 127.0.0.1:38628)
0925 14:49:11.015235 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.015247 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:11.015299 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:11.015521 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:11.015524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:11.015526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:11.015584 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.015599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.016586 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.016587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.017570 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.017572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.018527 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.018527 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.018600 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.018613 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.018678 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.018727 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260925 14:49:11.018815 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.015188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.21.205.195:46043)
0925 14:49:11.015363 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.015367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.015375 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:11.015406 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.015407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.015427 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.015510 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:11.015693 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.016498 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.017670 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.018352 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.018411 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.018495 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.018692 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.018727 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260925 14:49:11.019428 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.015273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:40405 (local address 127.0.0.1:58306)
0925 14:49:11.015388 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.015398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:11.015446 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:11.015929 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:11.015930 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:11.015932 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:11.015972 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.015973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.017187 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.017188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.018086 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.018088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.019177 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.019178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.019239 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.019243 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.019303 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.019347 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":26,"thread_start_us":64,"threads_started":1}
I20260925 14:49:11.019443 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.015390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58306 (local address 127.21.205.193:40405)
0925 14:49:11.015577 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.015580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.015601 (+ 21us) server_negotiation.cc:407] Connection header received
0925 14:49:11.015654 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.015714 (+ 60us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.015737 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.015872 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:11.016037 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.017077 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.018170 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.018992 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.019058 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.019131 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.019327 (+ 196us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.019369 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260925 14:49:11.019542 9163 tablet_service.cc:1467] Processing CreateTablet for tablet 63d86120c78942e9b51e9abb3d92c53b (DEFAULT_TABLE table=test-table [id=034aa82a0a3947eeab2ab0ead6963c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:11.019593 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.014910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38335 (local address 127.0.0.1:34952)
0925 14:49:11.015022 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.015057 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:11.015113 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:11.015829 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:11.015830 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:11.015833 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:11.015868 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.015869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.017202 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.017203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.018265 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.018268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.019388 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.019389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.019443 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.019447 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.019476 (+ 29us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.019517 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":20,"thread_start_us":68,"threads_started":1}
I20260925 14:49:11.019896 9163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d86120c78942e9b51e9abb3d92c53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:11.019972 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.015017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.21.205.194:38335)
0925 14:49:11.015638 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.015643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.015652 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:11.015693 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.015694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.015710 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.015945 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:11.016007 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.017074 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.018365 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.019304 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.019664 (+ 360us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.019822 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.019848 (+ 26us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.019889 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":101,"threads_started":1}
I20260925 14:49:11.020377 8899 tablet_service.cc:1467] Processing CreateTablet for tablet 63d86120c78942e9b51e9abb3d92c53b (DEFAULT_TABLE table=test-table [id=034aa82a0a3947eeab2ab0ead6963c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:11.020510 9031 tablet_service.cc:1467] Processing CreateTablet for tablet 63d86120c78942e9b51e9abb3d92c53b (DEFAULT_TABLE table=test-table [id=034aa82a0a3947eeab2ab0ead6963c8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:11.020679 8899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d86120c78942e9b51e9abb3d92c53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:11.020748 9031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63d86120c78942e9b51e9abb3d92c53b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:11.022897 9243 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Bootstrap starting.
I20260925 14:49:11.023209 9244 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Bootstrap starting.
I20260925 14:49:11.023510 9243 tablet_bootstrap.cc:654] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:11.023805 9245 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Bootstrap starting.
I20260925 14:49:11.023898 9244 tablet_bootstrap.cc:654] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:11.024016 9243 log.cc:824] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:11.024224 9244 log.cc:824] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:11.024410 9245 tablet_bootstrap.cc:654] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:11.024761 9243 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: No bootstrap required, opened a new log
I20260925 14:49:11.024761 9245 log.cc:824] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:11.024832 9243 ts_tablet_manager.cc:1397] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:49:11.024991 9244 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: No bootstrap required, opened a new log
I20260925 14:49:11.025063 9244 ts_tablet_manager.cc:1397] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260925 14:49:11.025494 9245 tablet_bootstrap.cc:492] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: No bootstrap required, opened a new log
I20260925 14:49:11.025626 9245 ts_tablet_manager.cc:1397] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:11.027053 9244 raft_consensus.cc:348] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.027185 9244 raft_consensus.cc:374] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:11.027205 9244 raft_consensus.cc:729] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561df3840174493c941d94173fab2984, State: Initialized, Role: FOLLOWER
I20260925 14:49:11.027323 9244 consensus_queue.cc:260] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.027590 9229 heartbeater.cc:503] Master 127.21.205.254:32979 was elected leader, sending a full tablet report...
I20260925 14:49:11.027614 9243 raft_consensus.cc:348] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.027817 9243 raft_consensus.cc:374] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:11.027865 9243 raft_consensus.cc:729] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd05621120fb4f29b94ab0f4627ffed2, State: Initialized, Role: FOLLOWER
I20260925 14:49:11.027971 9243 consensus_queue.cc:260] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.028221 9243 ts_tablet_manager.cc:1428] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260925 14:49:11.028246 9097 heartbeater.cc:503] Master 127.21.205.254:32979 was elected leader, sending a full tablet report...
I20260925 14:49:11.027581 9244 ts_tablet_manager.cc:1428] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 14:49:11.028885 9245 raft_consensus.cc:348] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.029066 9245 raft_consensus.cc:374] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:11.029102 9245 raft_consensus.cc:729] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e635926400d4995849c35db50ebf585, State: Initialized, Role: FOLLOWER
I20260925 14:49:11.029212 9245 consensus_queue.cc:260] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.029393 9245 ts_tablet_manager.cc:1428] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260925 14:49:11.029454 8965 heartbeater.cc:503] Master 127.21.205.254:32979 was elected leader, sending a full tablet report...
W20260925 14:49:11.120219 9098 tablet.cc:2369] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:49:11.246764 9230 tablet.cc:2369] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 14:49:11.247581 8966 tablet.cc:2369] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:49:11.306188 9249 raft_consensus.cc:482] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:11.306380 9249 raft_consensus.cc:504] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.306785 9249 leader_election.cc:288] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd05621120fb4f29b94ab0f4627ffed2 (127.21.205.194:38335), 0e635926400d4995849c35db50ebf585 (127.21.205.193:40405)
I20260925 14:49:11.309816 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.307056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:33067 (local address 127.21.205.194:38335)
0925 14:49:11.307137 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.307139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.307150 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:11.307185 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.307186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.307196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.307278 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:11.307496 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.308051 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309040 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.309421 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309479 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.309504 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.309701 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.309744 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 14:49:11.309818 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.306946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:38335 (local address 127.21.205.195:33067)
0925 14:49:11.307005 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.307018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:11.307084 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:11.307296 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:11.307297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:11.307299 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:11.307382 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.307384 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.308124 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.308125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.308941 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.308943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.309530 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.309610 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.309614 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.309672 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.309726 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:11.309868 9242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.307218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:37119 (local address 127.21.205.193:40405)
0925 14:49:11.307288 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.307290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.307388 (+ 98us) server_negotiation.cc:407] Connection header received
0925 14:49:11.307416 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.307418 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.307432 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.307492 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:11.307649 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.308214 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309124 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.309448 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309571 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.309580 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.309750 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.309791 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 14:49:11.310086 9252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.307103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:40405 (local address 127.21.205.195:37119)
0925 14:49:11.307277 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.307288 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:11.307345 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:11.307510 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:11.307511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:11.307513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:11.307577 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.307578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.308299 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.308300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.309040 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:11.309042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.309591 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.309592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:11.309650 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.309671 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 14:49:11.309729 (+ 58us) client_negotiation.cc:240] Negotiation successful
0925 14:49:11.309769 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":119,"threads_started":1}
I20260925 14:49:11.310220 9051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d86120c78942e9b51e9abb3d92c53b" candidate_uuid: "561df3840174493c941d94173fab2984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd05621120fb4f29b94ab0f4627ffed2" is_pre_election: true
I20260925 14:49:11.310400 9051 raft_consensus.cc:2454] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561df3840174493c941d94173fab2984 in term 0.
I20260925 14:49:11.310701 9117 leader_election.cc:302] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 561df3840174493c941d94173fab2984, dd05621120fb4f29b94ab0f4627ffed2; no voters:
I20260925 14:49:11.310801 8919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d86120c78942e9b51e9abb3d92c53b" candidate_uuid: "561df3840174493c941d94173fab2984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e635926400d4995849c35db50ebf585" is_pre_election: true
I20260925 14:49:11.310824 9249 raft_consensus.cc:2781] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:11.310909 9249 raft_consensus.cc:482] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:11.310940 9249 raft_consensus.cc:3037] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:11.310952 8919 raft_consensus.cc:2454] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 561df3840174493c941d94173fab2984 in term 0.
I20260925 14:49:11.311754 9249 raft_consensus.cc:504] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.311923 9249 leader_election.cc:288] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [CANDIDATE]: Term 1 election: Requested vote from peers dd05621120fb4f29b94ab0f4627ffed2 (127.21.205.194:38335), 0e635926400d4995849c35db50ebf585 (127.21.205.193:40405)
I20260925 14:49:11.312083 9051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d86120c78942e9b51e9abb3d92c53b" candidate_uuid: "561df3840174493c941d94173fab2984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd05621120fb4f29b94ab0f4627ffed2"
I20260925 14:49:11.312100 8919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63d86120c78942e9b51e9abb3d92c53b" candidate_uuid: "561df3840174493c941d94173fab2984" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e635926400d4995849c35db50ebf585"
I20260925 14:49:11.312176 8919 raft_consensus.cc:3037] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:11.312180 9051 raft_consensus.cc:3037] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:11.312929 8919 raft_consensus.cc:2454] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561df3840174493c941d94173fab2984 in term 1.
I20260925 14:49:11.313007 9051 raft_consensus.cc:2454] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 561df3840174493c941d94173fab2984 in term 1.
I20260925 14:49:11.313090 9117 leader_election.cc:302] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e635926400d4995849c35db50ebf585, 561df3840174493c941d94173fab2984; no voters:
I20260925 14:49:11.313215 9249 raft_consensus.cc:2781] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:11.313355 9249 raft_consensus.cc:686] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 1 LEADER]: Becoming Leader. State: Replica: 561df3840174493c941d94173fab2984, State: Running, Role: LEADER
I20260925 14:49:11.313448 9249 consensus_queue.cc:237] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } }
I20260925 14:49:11.314287 8776 catalog_manager.cc:5495] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 reported cstate change: term changed from 0 to 1, leader changed from <none> to 561df3840174493c941d94173fab2984 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "561df3840174493c941d94173fab2984" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561df3840174493c941d94173fab2984" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 46043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 } health_report { overall_health: UNKNOWN } } }
I20260925 14:49:11.374327 9240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.372225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.21.205.195:46043)
0925 14:49:11.372295 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:11.372297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:11.372312 (+ 15us) server_negotiation.cc:407] Connection header received
0925 14:49:11.372347 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:11.372348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:11.372366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:11.372427 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:11.372543 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.372997 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.373691 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:11.373881 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:11.373931 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:11.374053 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:11.374134 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:11.374202 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:11.374245 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 14:49:11.374253 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:49:11.376114 9051 raft_consensus.cc:1261] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 1 FOLLOWER]: Refusing update from remote peer 561df3840174493c941d94173fab2984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:11.376130 8919 raft_consensus.cc:1261] T 63d86120c78942e9b51e9abb3d92c53b P 0e635926400d4995849c35db50ebf585 [term 1 FOLLOWER]: Refusing update from remote peer 561df3840174493c941d94173fab2984: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:11.376477 9253 consensus_queue.cc:1035] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd05621120fb4f29b94ab0f4627ffed2" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 38335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:11.376624 9249 consensus_queue.cc:1035] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e635926400d4995849c35db50ebf585" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 40405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:11.378763 9256 mvcc.cc:205] Tried to move back new op lower bound from 7333264389633851392 to 7333264389380562944. Current Snapshot: MvccSnapshot[applied={T|T < 7333264389633851392}]
I20260925 14:49:11.379580 9258 mvcc.cc:205] Tried to move back new op lower bound from 7333264389633851392 to 7333264389380562944. Current Snapshot: MvccSnapshot[applied={T|T < 7333264389633851392}]
I20260925 14:49:11.381186 8776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:11.381278 8776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:11.381995 8776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ea352d635e343b3ba9cacb8dfee3be1: Sending DeleteTablet for 3 replicas of tablet 63d86120c78942e9b51e9abb3d92c53b
I20260925 14:49:11.382328 9031 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d86120c78942e9b51e9abb3d92c53b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:11 UTC) from {username='slave'} at 127.0.0.1:34952
I20260925 14:49:11.382376 8899 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d86120c78942e9b51e9abb3d92c53b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:11 UTC) from {username='slave'} at 127.0.0.1:58306
I20260925 14:49:11.382560 9265 tablet_replica.cc:330] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: stopping tablet replica
I20260925 14:49:11.382658 9265 raft_consensus.cc:2229] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:11.382679 9163 tablet_service.cc:1514] Processing DeleteTablet for tablet 63d86120c78942e9b51e9abb3d92c53b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:11 UTC) from {username='slave'} at 127.0.0.1:38628
I20260925 14:49:11.382766 9265 raft_consensus.cc:2258] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:11.382807 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8837
I20260925 14:49:11.384730 9267 tablet_replica.cc:330] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: stopping tablet replica
I20260925 14:49:11.384816 9265 ts_tablet_manager.cc:1905] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:11.384932 9267 raft_consensus.cc:2229] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:49:11.385099 9267 raft_consensus.cc:2258] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:11.385541 9267 ts_tablet_manager.cc:1905] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:11.386555 9265 ts_tablet_manager.cc:1918] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:11.386631 9265 log.cc:1196] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-1/wal/wals/63d86120c78942e9b51e9abb3d92c53b
I20260925 14:49:11.387123 9265 ts_tablet_manager.cc:1939] T 63d86120c78942e9b51e9abb3d92c53b P dd05621120fb4f29b94ab0f4627ffed2: Deleting consensus metadata
I20260925 14:49:11.387805 8761 catalog_manager.cc:4841] TS dd05621120fb4f29b94ab0f4627ffed2 (127.21.205.194:38335): tablet 63d86120c78942e9b51e9abb3d92c53b (table test-table [id=034aa82a0a3947eeab2ab0ead6963c8c]) successfully deleted
I20260925 14:49:11.387845 9267 ts_tablet_manager.cc:1918] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:11.388013 9267 log.cc:1196] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/63d86120c78942e9b51e9abb3d92c53b
I20260925 14:49:11.388362 9267 ts_tablet_manager.cc:1939] T 63d86120c78942e9b51e9abb3d92c53b P 561df3840174493c941d94173fab2984: Deleting consensus metadata
I20260925 14:49:11.388793 8761 catalog_manager.cc:4841] TS 561df3840174493c941d94173fab2984 (127.21.205.195:46043): tablet 63d86120c78942e9b51e9abb3d92c53b (table test-table [id=034aa82a0a3947eeab2ab0ead6963c8c]) successfully deleted
W20260925 14:49:11.390784 8761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:40405 (error 108)
I20260925 14:49:11.390928 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8968
I20260925 14:49:11.392758 9241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:11.392590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:40405 (local address 127.0.0.1:58322)
0925 14:49:11.392665 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:11.392716 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:40405: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 14:49:11.392889 8761 catalog_manager.cc:4584] TS 0e635926400d4995849c35db50ebf585 (127.21.205.193:40405): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 63d86120c78942e9b51e9abb3d92c53b: Network error: Client connection negotiation failed: client connection to 127.21.205.193:40405: connect: Connection refused (error 111)
I20260925 14:49:11.396399 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9100
I20260925 14:49:11.401273 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 8745
2026-09-25T14:49:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (969 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:11 dist-test-slave-d7lw krb5kdc[9271](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:11 dist-test-slave-d7lw krb5kdc[9271](info): set up 2 sockets
Sep 25 14:49:11 dist-test-slave-d7lw krb5kdc[9271](info): commencing operation
krb5kdc: starting...
W20260925 14:49:12.422703 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.002s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:12.535725 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:44953
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:44953
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:12.615898 9284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:12.616170 9284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:12.616211 9284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:12.617435 9284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:12.617511 9284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:12.617539 9284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:12.617595 9284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:12.617626 9284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:12.619127 9284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:44953
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:44953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:12.619442 9284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:12.619709 9284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:12.624557 9289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:12.624585 9290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:12.624604 9292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:12.624807 9284 server_base.cc:1034] running on GCE node
I20260925 14:49:12.625063 9284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:12.625392 9284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:12.626574 9284 hybrid_clock.cc:648] HybridClock initialized: now 1790347752626558 us; error 24 us; skew 500 ppm
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:12.629207 9284 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:12.629686 9284 webserver.cc:460] Webserver started at http://127.21.205.254:36029/ using document root <none> and password file <none>
I20260925 14:49:12.629921 9284 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:12.629989 9284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:12.630146 9284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:12.631202 9284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "4cdab23f717842b0a80dca3eb8360669"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "aaffd1052b020798d49ea86f0d0a40be"
server_key_iv: "0d7fafc7dbcaebdece194f69d311d186"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.631574 9284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "4cdab23f717842b0a80dca3eb8360669"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "aaffd1052b020798d49ea86f0d0a40be"
server_key_iv: "0d7fafc7dbcaebdece194f69d311d186"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.632931 9284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:12.633651 9300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.633823 9284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:12.633926 9284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "4cdab23f717842b0a80dca3eb8360669"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "aaffd1052b020798d49ea86f0d0a40be"
server_key_iv: "0d7fafc7dbcaebdece194f69d311d186"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.634008 9284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:12.651408 9284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:12.653762 9284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:12.653927 9284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:12.658260 9284 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:44953
I20260925 14:49:12.658308 9353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:44953 every 8 connection(s)
I20260925 14:49:12.658669 9284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:12.659503 9354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:12.660619 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9284
I20260925 14:49:12.660738 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:12.660948 22327 external_mini_cluster.cc:1428] Setting key 80d5fb2f01282db2feb4824527206a94
I20260925 14:49:12.662415 9354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: Bootstrap starting.
I20260925 14:49:12.663014 9354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:12.663314 9354 log.cc:824] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:12.664069 9354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: No bootstrap required, opened a new log
I20260925 14:49:12.665844 9354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } }
I20260925 14:49:12.666002 9354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:12.666067 9354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdab23f717842b0a80dca3eb8360669, State: Initialized, Role: FOLLOWER
I20260925 14:49:12.666231 9354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } }
I20260925 14:49:12.666332 9354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:12.666368 9354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:12.666417 9354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:12.667186 9354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } }
I20260925 14:49:12.667309 9354 leader_election.cc:302] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cdab23f717842b0a80dca3eb8360669; no voters:
I20260925 14:49:12.667501 9354 leader_election.cc:288] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:12.667562 9359 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:12.667742 9359 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [term 1 LEADER]: Becoming Leader. State: Replica: 4cdab23f717842b0a80dca3eb8360669, State: Running, Role: LEADER
I20260925 14:49:12.667796 9354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:12.667939 9359 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } }
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:12.669054 9361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cdab23f717842b0a80dca3eb8360669. Latest consensus state: current_term: 1 leader_uuid: "4cdab23f717842b0a80dca3eb8360669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } } }
I20260925 14:49:12.669147 9361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:12.669438 9374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:12.669690 9360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cdab23f717842b0a80dca3eb8360669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdab23f717842b0a80dca3eb8360669" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44953 } } }
I20260925 14:49:12.670929 9360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:12.671098 9374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:12.672081 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:12.661474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49442 (local address 127.21.205.254:44953)
0925 14:49:12.661667 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:12.661681 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:12.661705 (+ 24us) server_negotiation.cc:407] Connection header received
0925 14:49:12.662232 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:12.662239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:12.662285 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:12.662433 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:12.663086 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.664337 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.665099 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.665521 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.665647 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:12.668744 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:12.668748 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:12.668748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:12.668761 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:12.670517 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:12.670912 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:12.670915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:12.670915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:12.671035 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:12.671253 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:12.671254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:12.671254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:12.671492 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:12.671649 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:12.671757 (+ 108us) server_negotiation.cc:299] Negotiation successful
0925 14:49:12.671928 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260925 14:49:12.672636 9374 catalog_manager.cc:1347] Generated new cluster ID: 8a7455545c664d7a99204194004a1ed3
I20260925 14:49:12.672688 9374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:12.700668 9374 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:12.701488 9374 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:12.705204 9374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: Generated new TSK 0
I20260925 14:49:12.705381 9374 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.205.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
Entry for principal kudu/127.21.205.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab.
I20260925 14:49:12.734006 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:12.814689 9380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:12.814909 9380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:12.814949 9380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:12.816138 9380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:12.816196 9380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:12.816239 9380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:12.817688 9380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.193.keytab
--principal=kudu/127.21.205.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:12.817943 9380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:12.818168 9380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:12.823083 9386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:12.823072 9388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:12.823071 9385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:12.823321 9380 server_base.cc:1034] running on GCE node
I20260925 14:49:12.823557 9380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:12.823839 9380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:12.825001 9380 hybrid_clock.cc:648] HybridClock initialized: now 1790347752824981 us; error 35 us; skew 500 ppm
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:12.827600 9380 init.cc:377] Logged in from keytab as kudu/127.21.205.193@KRBTEST.COM (short username kudu)
I20260925 14:49:12.828061 9380 webserver.cc:460] Webserver started at http://127.21.205.193:38909/ using document root <none> and password file <none>
I20260925 14:49:12.828240 9380 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:12.828282 9380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:12.828423 9380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:12.829418 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "15b17484c1c849819e071e7f40f4f18b"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "a7c596cfce8208ff8b587d521b08fcd5"
server_key_iv: "95cbe302a9cee858de6986b1be014638"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.829733 9380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "15b17484c1c849819e071e7f40f4f18b"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "a7c596cfce8208ff8b587d521b08fcd5"
server_key_iv: "95cbe302a9cee858de6986b1be014638"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.830993 9380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:49:12.831650 9396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.831866 9380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:12.831941 9380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "15b17484c1c849819e071e7f40f4f18b"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "a7c596cfce8208ff8b587d521b08fcd5"
server_key_iv: "95cbe302a9cee858de6986b1be014638"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.831997 9380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:12.845584 9380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:12.847914 9380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:12.848038 9380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:12.848219 9380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:12.848462 9380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:12.848495 9380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.848518 9380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:12.848549 9380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.854493 9380 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:33649
I20260925 14:49:12.854568 9510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:33649 every 8 connection(s)
I20260925 14:49:12.854895 9380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:49:12.858929 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9380
I20260925 14:49:12.859077 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:12.859293 22327 external_mini_cluster.cc:1428] Setting key 8defbce5e4a822d5a17257783122d6ff
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.193@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:12.864997 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:12.855592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:40893 (local address 127.21.205.254:44953)
0925 14:49:12.855667 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:12.855669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:12.855932 (+ 263us) server_negotiation.cc:407] Connection header received
0925 14:49:12.856451 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:12.856452 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:12.856465 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:12.856533 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:12.857512 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.858055 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.859351 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.859699 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.859775 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:12.862128 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:12.862130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:12.862130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:12.862139 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:12.863543 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:12.864031 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:12.864033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:12.864034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:12.864093 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:12.864353 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:12.864354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:12.864354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:12.864466 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:12.864536 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:12.864824 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 14:49:12.864909 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 14:49:12.865167 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:12.855568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:44953 (local address 127.21.205.193:40893)
0925 14:49:12.855761 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:12.855840 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:12.856345 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:12.856611 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:12.856614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:12.857219 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:12.857407 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:12.857412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.858176 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.858177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:12.859233 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:12.859236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:12.859829 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:12.859830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:12.859981 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:12.860464 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:12.860474 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:12.862033 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:12.863638 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:12.863645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:12.863648 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:12.863942 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:12.864160 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:12.864161 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:12.864162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:12.864276 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:12.864540 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:12.864546 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:12.864680 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 14:49:12.864789 (+ 109us) client_negotiation.cc:240] Negotiation successful
0925 14:49:12.864939 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260925 14:49:12.865635 9511 heartbeater.cc:347] Connected to a master server at 127.21.205.254:44953
I20260925 14:49:12.865727 9511 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:12.865916 9511 heartbeater.cc:511] Master 127.21.205.254:44953 requested a full tablet report, sending...
I20260925 14:49:12.866472 9317 ts_manager.cc:194] Registered new tserver with Master: 15b17484c1c849819e071e7f40f4f18b (127.21.205.193:33649)
I20260925 14:49:12.867434 9317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.193@KRBTEST.COM'} at 127.21.205.193:40893
WARNING: no policy specified for kudu/127.21.205.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
Entry for principal kudu/127.21.205.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab.
I20260925 14:49:12.884799 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:12.965835 9516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:12.966065 9516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:12.966091 9516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:12.967216 9516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:12.967252 9516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:12.967278 9516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:12.968950 9516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.194.keytab
--principal=kudu/127.21.205.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:12.969192 9516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:12.969414 9516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:12.974573 9521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:12.974536 9522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:12.974735 9516 server_base.cc:1034] running on GCE node
W20260925 14:49:12.974561 9524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:12.975092 9516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:12.975395 9516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:12.976565 9516 hybrid_clock.cc:648] HybridClock initialized: now 1790347752976549 us; error 47 us; skew 500 ppm
Sep 25 14:49:12 dist-test-slave-d7lw krb5kdc[9271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:12.979305 9516 init.cc:377] Logged in from keytab as kudu/127.21.205.194@KRBTEST.COM (short username kudu)
I20260925 14:49:12.979772 9516 webserver.cc:460] Webserver started at http://127.21.205.194:39977/ using document root <none> and password file <none>
I20260925 14:49:12.980067 9516 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:12.980134 9516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:12.980290 9516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:12.981389 9516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "433e97ac46ed420c86262ced8db90a59"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "8634d466bdb9a5dba2a3c021cfd2b085"
server_key_iv: "ee29f58055f395c967573ca568cabe6a"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.981768 9516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "433e97ac46ed420c86262ced8db90a59"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "8634d466bdb9a5dba2a3c021cfd2b085"
server_key_iv: "ee29f58055f395c967573ca568cabe6a"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.983150 9516 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:12.983948 9531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.984129 9516 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:12.984232 9516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "433e97ac46ed420c86262ced8db90a59"
format_stamp: "Formatted at 2026-09-25 14:49:12 on dist-test-slave-d7lw"
server_key: "8634d466bdb9a5dba2a3c021cfd2b085"
server_key_iv: "ee29f58055f395c967573ca568cabe6a"
server_key_version: "encryptionkey@0"
I20260925 14:49:12.984326 9516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:12.995905 9516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:12.998400 9516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:12.998579 9516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:12.998843 9516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:12.999171 9516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:12.999226 9516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:12.999284 9516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:12.999308 9516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.005717 9516 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:33353
I20260925 14:49:13.005826 9645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:33353 every 8 connection(s)
I20260925 14:49:13.006136 9516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:49:13.009338 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9516
I20260925 14:49:13.009440 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:13.009646 22327 external_mini_cluster.cc:1428] Setting key ac1efe4c97938ff18889ea0be5f89aaf
Sep 25 14:49:13 dist-test-slave-d7lw krb5kdc[9271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.194@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:13.017180 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.007000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:43725 (local address 127.21.205.254:44953)
0925 14:49:13.007063 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.007065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.007291 (+ 226us) server_negotiation.cc:407] Connection header received
0925 14:49:13.007860 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.007862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.007875 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.007939 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.008924 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.009501 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.010758 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.010971 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.011058 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.013732 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.013734 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.013734 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:13.013743 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.015577 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.016094 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.016095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.016096 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.016164 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.016427 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.016429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.016429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.016594 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.016667 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.016951 (+ 284us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.017051 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:49:13.017202 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.006977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:44953 (local address 127.21.205.194:43725)
0925 14:49:13.007158 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.007196 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.007719 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.008018 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.008021 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.008607 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:13.008820 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.008824 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.009620 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.009621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.010664 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.010667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.011069 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.011070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.011197 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.011827 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:13.011838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:13.013614 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:13.015680 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:13.015687 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:13.015690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:13.016011 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:13.016240 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:13.016240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:13.016241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:13.016349 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:13.016663 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:13.016667 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:13.016798 (+ 131us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.016879 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.017017 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260925 14:49:13.017923 9646 heartbeater.cc:347] Connected to a master server at 127.21.205.254:44953
I20260925 14:49:13.018052 9646 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:13.018256 9646 heartbeater.cc:511] Master 127.21.205.254:44953 requested a full tablet report, sending...
I20260925 14:49:13.018679 9317 ts_manager.cc:194] Registered new tserver with Master: 433e97ac46ed420c86262ced8db90a59 (127.21.205.194:33353)
WARNING: no policy specified for kudu/127.21.205.195@KRBTEST.COM; defaulting to no policy
I20260925 14:49:13.019098 9317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.194@KRBTEST.COM'} at 127.21.205.194:43725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
Entry for principal kudu/127.21.205.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab.
I20260925 14:49:13.034940 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:13.119470 9651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:13.119750 9651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:13.119791 9651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:13.121073 9651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:13.121152 9651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:13.121236 9651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:13.122865 9651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.195.keytab
--principal=kudu/127.21.205.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:44953
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:13.123171 9651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:13.123435 9651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:13.128865 9657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.128873 9656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.128980 9651 server_base.cc:1034] running on GCE node
W20260925 14:49:13.128901 9659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.129420 9651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:13.129702 9651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:13.130889 9651 hybrid_clock.cc:648] HybridClock initialized: now 1790347753130877 us; error 30 us; skew 500 ppm
Sep 25 14:49:13 dist-test-slave-d7lw krb5kdc[9271](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347753, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:13.133483 9651 init.cc:377] Logged in from keytab as kudu/127.21.205.195@KRBTEST.COM (short username kudu)
I20260925 14:49:13.134016 9651 webserver.cc:460] Webserver started at http://127.21.205.195:41885/ using document root <none> and password file <none>
I20260925 14:49:13.134265 9651 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:13.134311 9651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:13.134528 9651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:13.135666 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2ad21a97e292c3427bb24c71b9decca0"
server_key_iv: "955c2845474e6c9d77b15d172602ef48"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.136140 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2ad21a97e292c3427bb24c71b9decca0"
server_key_iv: "955c2845474e6c9d77b15d172602ef48"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.137668 9651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:13.138496 9666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.138689 9651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:13.138799 9651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2ad21a97e292c3427bb24c71b9decca0"
server_key_iv: "955c2845474e6c9d77b15d172602ef48"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.138890 9651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:13.162397 9651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:13.164685 9651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:13.164872 9651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:13.165123 9651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:13.165465 9651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:13.165521 9651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.165556 9651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:13.165583 9651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.172343 9651 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:39765
I20260925 14:49:13.172417 9780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:39765 every 8 connection(s)
I20260925 14:49:13.172780 9651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
Sep 25 14:49:13 dist-test-slave-d7lw krb5kdc[9271](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347753, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.195@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:13.180579 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9651
I20260925 14:49:13.180671 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:13.180876 22327 external_mini_cluster.cc:1428] Setting key 00f830bdc8b8e9685198665b93f4e68a
I20260925 14:49:13.182448 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.173563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:56091 (local address 127.21.205.254:44953)
0925 14:49:13.173654 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.173657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.173856 (+ 199us) server_negotiation.cc:407] Connection header received
0925 14:49:13.174351 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.174352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.174362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.174415 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.175390 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.175991 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.177012 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.177209 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.177261 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.179408 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.179409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.179410 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:13.179416 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.180989 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.181460 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.181461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.181462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.181528 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.181829 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.181830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.181830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.181984 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.182047 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.182311 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.182381 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 14:49:13.182704 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.173531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:44953 (local address 127.21.205.195:56091)
0925 14:49:13.173725 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.173764 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.174260 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.174472 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.174476 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.175083 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:13.175300 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.175306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.176083 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.176083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.176929 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.176932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.177285 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.177286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.177396 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.177912 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:13.177920 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:13.179317 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:13.181077 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:13.181086 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:13.181089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:13.181386 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:13.181601 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:13.181602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:13.181602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:13.181760 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:13.182050 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:13.182055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:13.182193 (+ 138us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.182284 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.182417 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260925 14:49:13.183180 9781 heartbeater.cc:347] Connected to a master server at 127.21.205.254:44953
I20260925 14:49:13.183286 9781 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:13.183495 9781 heartbeater.cc:511] Master 127.21.205.254:44953 requested a full tablet report, sending...
I20260925 14:49:13.183928 9317 ts_manager.cc:194] Registered new tserver with Master: dfa3d38424fa4c1cbcb29d6cfca63fd9 (127.21.205.195:39765)
I20260925 14:49:13.184329 9317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.205.195@KRBTEST.COM'} at 127.21.205.195:56091
I20260925 14:49:13.192780 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:13.200407 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.194162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.21.205.254:44953)
0925 14:49:13.194228 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.194230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.194349 (+ 119us) server_negotiation.cc:407] Connection header received
0925 14:49:13.194444 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.194445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.194455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.194507 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.195070 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.195585 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.196400 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.196614 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.196705 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.197886 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.197888 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.197888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:13.197896 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.199057 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.199425 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.199426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.199426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.199490 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:13.199697 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:13.199698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:13.199698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:13.199943 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.200056 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.200203 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.200298 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:49:13.201954 9317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:13.203511 9317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:13.210141 9790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.205788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:33649 (local address 127.0.0.1:56554)
0925 14:49:13.205964 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.206008 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.206080 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.206561 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.206562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.206564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.206624 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.206626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.207635 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.207637 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.209073 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.209076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.209807 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.209892 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.209907 (+ 15us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.209976 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.210041 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":22,"thread_start_us":59,"threads_started":1}
I20260925 14:49:13.210337 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.206163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36370 (local address 127.21.205.195:39765)
0925 14:49:13.206352 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.206357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.206486 (+ 129us) server_negotiation.cc:407] Connection header received
0925 14:49:13.206527 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.206529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.206547 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.206824 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.206899 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.208001 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209143 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.209875 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209968 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.210030 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.210188 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.210238 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":115,"threads_started":1}
I20260925 14:49:13.210367 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.205899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.21.205.193:33649)
0925 14:49:13.206391 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.206395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.206404 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:13.206440 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.206442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.206457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.206556 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.206751 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.207528 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209178 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.209716 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209810 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.209876 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.210096 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.210161 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":93,"threads_started":1}
I20260925 14:49:13.210505 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.205977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:39765 (local address 127.0.0.1:36370)
0925 14:49:13.206363 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.206373 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.206427 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.206704 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.206705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.206707 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.206745 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.206747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.208083 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.208085 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.209044 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.209047 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.209964 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.210017 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.210020 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.210087 (+ 67us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.210121 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":3,"thread_start_us":36,"threads_started":1}
I20260925 14:49:13.210599 9792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.205788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:33353 (local address 127.0.0.1:55346)
0925 14:49:13.206173 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.206183 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.206258 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.206466 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.206470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.206473 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.206559 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.206562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.208017 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.208018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.209159 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.209161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.210005 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.210006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.210078 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.210083 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.210117 (+ 34us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.210162 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":33,"thread_start_us":58,"threads_started":1}
I20260925 14:49:13.210973 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.205998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55346 (local address 127.21.205.194:33353)
0925 14:49:13.206160 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.206166 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.206235 (+ 69us) server_negotiation.cc:407] Connection header received
0925 14:49:13.206330 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.206333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.206351 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.206433 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.206663 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.207913 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.209226 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.209910 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.210213 (+ 303us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.210810 (+ 597us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.210838 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.210888 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260925 14:49:13.211210 9444 tablet_service.cc:1467] Processing CreateTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 (DEFAULT_TABLE table=test-table [id=c40d9f9b93a64eee8386842773e1c78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.211261 9714 tablet_service.cc:1467] Processing CreateTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 (DEFAULT_TABLE table=test-table [id=c40d9f9b93a64eee8386842773e1c78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.211615 9444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe3ec78e97e4ff5a2bc618d797575d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.211618 9714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe3ec78e97e4ff5a2bc618d797575d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.211599 9579 tablet_service.cc:1467] Processing CreateTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 (DEFAULT_TABLE table=test-table [id=c40d9f9b93a64eee8386842773e1c78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.211951 9579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afe3ec78e97e4ff5a2bc618d797575d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.214365 9795 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: Bootstrap starting.
I20260925 14:49:13.215101 9796 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Bootstrap starting.
I20260925 14:49:13.215265 9795 tablet_bootstrap.cc:654] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.215840 9796 tablet_bootstrap.cc:654] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.215961 9795 log.cc:824] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.215971 9797 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Bootstrap starting.
I20260925 14:49:13.216439 9796 log.cc:824] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.216647 9797 tablet_bootstrap.cc:654] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.216852 9795 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: No bootstrap required, opened a new log
I20260925 14:49:13.216933 9795 ts_tablet_manager.cc:1397] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260925 14:49:13.217170 9796 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: No bootstrap required, opened a new log
I20260925 14:49:13.217214 9796 ts_tablet_manager.cc:1397] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260925 14:49:13.217206 9797 log.cc:824] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.218078 9797 tablet_bootstrap.cc:492] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: No bootstrap required, opened a new log
I20260925 14:49:13.218151 9797 ts_tablet_manager.cc:1397] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:13.218724 9795 raft_consensus.cc:348] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.218868 9795 raft_consensus.cc:374] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.218889 9795 raft_consensus.cc:729] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b17484c1c849819e071e7f40f4f18b, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.219012 9795 consensus_queue.cc:260] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.219300 9511 heartbeater.cc:503] Master 127.21.205.254:44953 was elected leader, sending a full tablet report...
I20260925 14:49:13.219300 9795 ts_tablet_manager.cc:1428] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260925 14:49:13.219256 9796 raft_consensus.cc:348] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.219422 9796 raft_consensus.cc:374] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.219456 9796 raft_consensus.cc:729] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 433e97ac46ed420c86262ced8db90a59, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.219574 9796 consensus_queue.cc:260] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.219980 9796 ts_tablet_manager.cc:1428] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 14:49:13.220108 9646 heartbeater.cc:503] Master 127.21.205.254:44953 was elected leader, sending a full tablet report...
I20260925 14:49:13.220615 9797 raft_consensus.cc:348] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.220798 9797 raft_consensus.cc:374] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.220844 9797 raft_consensus.cc:729] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfa3d38424fa4c1cbcb29d6cfca63fd9, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.221004 9797 consensus_queue.cc:260] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.221315 9781 heartbeater.cc:503] Master 127.21.205.254:44953 was elected leader, sending a full tablet report...
I20260925 14:49:13.221297 9797 ts_tablet_manager.cc:1428] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260925 14:49:13.255013 9803 raft_consensus.cc:482] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:13.255156 9803 raft_consensus.cc:504] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.255580 9803 leader_election.cc:288] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 433e97ac46ed420c86262ced8db90a59 (127.21.205.194:33353), 15b17484c1c849819e071e7f40f4f18b (127.21.205.193:33649)
W20260925 14:49:13.256721 9647 tablet.cc:2369] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:49:13.258857 9793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.255898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:37041 (local address 127.21.205.194:33353)
0925 14:49:13.255961 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.255963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.256110 (+ 147us) server_negotiation.cc:407] Connection header received
0925 14:49:13.256189 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.256190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.256204 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.256258 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.256448 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.256992 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258102 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.258469 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258528 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.258562 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.258732 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.258785 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 14:49:13.258867 9791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.256012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:49691 (local address 127.21.205.193:33649)
0925 14:49:13.256070 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.256072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.256081 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:13.256114 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.256116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.256128 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.256191 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.256401 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.257100 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258068 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.258477 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258568 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.258585 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.258765 (+ 180us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.258805 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260925 14:49:13.258862 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.255764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:33649 (local address 127.21.205.195:49691)
0925 14:49:13.255858 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.255873 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.255922 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.256213 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.256214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.256216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.256316 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.256318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.257189 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.257190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.257964 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.257966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.258587 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.258665 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.258668 (+ 3us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.258739 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.258784 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":28}
I20260925 14:49:13.258873 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.255780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:33353 (local address 127.21.205.195:37041)
0925 14:49:13.256027 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.256036 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.256105 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.256283 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.256284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.256285 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.256346 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.256347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.257086 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.257087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.257980 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.257982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.258562 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.258563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.258652 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.258656 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.258701 (+ 45us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.258770 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":28,"thread_start_us":135,"threads_started":1}
I20260925 14:49:13.259349 9599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe3ec78e97e4ff5a2bc618d797575d2" candidate_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433e97ac46ed420c86262ced8db90a59" is_pre_election: true
I20260925 14:49:13.259436 9464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe3ec78e97e4ff5a2bc618d797575d2" candidate_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b17484c1c849819e071e7f40f4f18b" is_pre_election: true
I20260925 14:49:13.259548 9599 raft_consensus.cc:2454] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa3d38424fa4c1cbcb29d6cfca63fd9 in term 0.
I20260925 14:49:13.259552 9464 raft_consensus.cc:2454] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfa3d38424fa4c1cbcb29d6cfca63fd9 in term 0.
I20260925 14:49:13.259843 9667 leader_election.cc:302] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433e97ac46ed420c86262ced8db90a59, dfa3d38424fa4c1cbcb29d6cfca63fd9; no voters:
I20260925 14:49:13.260035 9803 raft_consensus.cc:2781] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:13.260076 9803 raft_consensus.cc:482] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:13.260095 9803 raft_consensus.cc:3037] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:13.260810 9803 raft_consensus.cc:504] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.260972 9803 leader_election.cc:288] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [CANDIDATE]: Term 1 election: Requested vote from peers 433e97ac46ed420c86262ced8db90a59 (127.21.205.194:33353), 15b17484c1c849819e071e7f40f4f18b (127.21.205.193:33649)
I20260925 14:49:13.261143 9599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe3ec78e97e4ff5a2bc618d797575d2" candidate_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "433e97ac46ed420c86262ced8db90a59"
I20260925 14:49:13.261155 9464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afe3ec78e97e4ff5a2bc618d797575d2" candidate_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b17484c1c849819e071e7f40f4f18b"
I20260925 14:49:13.261237 9599 raft_consensus.cc:3037] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:13.261236 9464 raft_consensus.cc:3037] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:13.262169 9599 raft_consensus.cc:2454] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa3d38424fa4c1cbcb29d6cfca63fd9 in term 1.
I20260925 14:49:13.262169 9464 raft_consensus.cc:2454] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfa3d38424fa4c1cbcb29d6cfca63fd9 in term 1.
I20260925 14:49:13.262356 9667 leader_election.cc:302] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 433e97ac46ed420c86262ced8db90a59, dfa3d38424fa4c1cbcb29d6cfca63fd9; no voters:
I20260925 14:49:13.262506 9803 raft_consensus.cc:2781] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:13.262687 9803 raft_consensus.cc:686] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 1 LEADER]: Becoming Leader. State: Replica: dfa3d38424fa4c1cbcb29d6cfca63fd9, State: Running, Role: LEADER
I20260925 14:49:13.262810 9803 consensus_queue.cc:237] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } }
I20260925 14:49:13.263747 9316 catalog_manager.cc:5495] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfa3d38424fa4c1cbcb29d6cfca63fd9 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfa3d38424fa4c1cbcb29d6cfca63fd9" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 39765 } health_report { overall_health: HEALTHY } } }
I20260925 14:49:13.272478 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.270186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36372 (local address 127.21.205.195:39765)
0925 14:49:13.270249 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.270251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.270259 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:13.270298 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.270299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.270317 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.270373 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:13.270477 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.270944 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.271670 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.271932 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.271989 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.272130 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:13.272233 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:13.272307 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.272320 (+ 13us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.272341 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:49:13.274432 9464 raft_consensus.cc:1261] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b [term 1 FOLLOWER]: Refusing update from remote peer dfa3d38424fa4c1cbcb29d6cfca63fd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:13.274431 9599 raft_consensus.cc:1261] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 1 FOLLOWER]: Refusing update from remote peer dfa3d38424fa4c1cbcb29d6cfca63fd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:13.274785 9803 consensus_queue.cc:1035] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "433e97ac46ed420c86262ced8db90a59" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 33353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:13.274935 9805 consensus_queue.cc:1035] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15b17484c1c849819e071e7f40f4f18b" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 33649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:13.277738 9808 mvcc.cc:205] Tried to move back new op lower bound from 7333264397409230848 to 7333264397365256192. Current Snapshot: MvccSnapshot[applied={T|T < 7333264397409230848}]
I20260925 14:49:13.278082 9810 mvcc.cc:205] Tried to move back new op lower bound from 7333264397409230848 to 7333264397365256192. Current Snapshot: MvccSnapshot[applied={T|T < 7333264397409230848}]
I20260925 14:49:13.278769 9809 mvcc.cc:205] Tried to move back new op lower bound from 7333264397409230848 to 7333264397365256192. Current Snapshot: MvccSnapshot[applied={T|T < 7333264397409230848 or (T in {7333264397409230848})}]
I20260925 14:49:13.280638 9316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:13.280737 9316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:13.281599 9316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4cdab23f717842b0a80dca3eb8360669: Sending DeleteTablet for 3 replicas of tablet afe3ec78e97e4ff5a2bc618d797575d2
I20260925 14:49:13.281948 9579 tablet_service.cc:1514] Processing DeleteTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:13 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:55346
I20260925 14:49:13.282016 9444 tablet_service.cc:1514] Processing DeleteTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:13 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:56554
I20260925 14:49:13.282307 9819 tablet_replica.cc:330] T afe3ec78e97e4ff5a2bc618d797575d2 P 15b17484c1c849819e071e7f40f4f18b: stopping tablet replica
I20260925 14:49:13.282380 9714 tablet_service.cc:1514] Processing DeleteTablet for tablet afe3ec78e97e4ff5a2bc618d797575d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:13 UTC) from {username='kudu', principal='kudu/127.21.205.254@KRBTEST.COM'} at 127.0.0.1:36370
I20260925 14:49:13.282500 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9380
I20260925 14:49:13.283772 9820 tablet_replica.cc:330] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: stopping tablet replica
I20260925 14:49:13.283964 9820 raft_consensus.cc:2229] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:49:13.284183 9820 raft_consensus.cc:2258] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:13.284775 9820 ts_tablet_manager.cc:1905] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:13.286715 9820 ts_tablet_manager.cc:1918] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:13.286865 9820 log.cc:1196] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/afe3ec78e97e4ff5a2bc618d797575d2
I20260925 14:49:13.287235 9820 ts_tablet_manager.cc:1939] T afe3ec78e97e4ff5a2bc618d797575d2 P dfa3d38424fa4c1cbcb29d6cfca63fd9: Deleting consensus metadata
I20260925 14:49:13.287596 9818 tablet_replica.cc:330] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: stopping tablet replica
I20260925 14:49:13.287822 9818 raft_consensus.cc:2229] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:13.288018 9818 raft_consensus.cc:2258] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:13.288627 9818 ts_tablet_manager.cc:1905] T afe3ec78e97e4ff5a2bc618d797575d2 P 433e97ac46ed420c86262ced8db90a59: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260925 14:49:13.289212 9303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:33649 (error 108)
I20260925 14:49:13.289318 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9516
I20260925 14:49:13.291044 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.290887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:33649 (local address 127.0.0.1:56570)
0925 14:49:13.290949 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.290992 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:33649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 14:49:13.291206 9303 catalog_manager.cc:4584] TS 15b17484c1c849819e071e7f40f4f18b (127.21.205.193:33649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afe3ec78e97e4ff5a2bc618d797575d2: Network error: Client connection negotiation failed: client connection to 127.21.205.193:33649: connect: Connection refused (error 111)
I20260925 14:49:13.291848 9303 catalog_manager.cc:4841] TS dfa3d38424fa4c1cbcb29d6cfca63fd9 (127.21.205.195:39765): tablet afe3ec78e97e4ff5a2bc618d797575d2 (table test-table [id=c40d9f9b93a64eee8386842773e1c78f]) successfully deleted
I20260925 14:49:13.295467 9789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.295313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:33353 (local address 127.0.0.1:55358)
0925 14:49:13.295386 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.295420 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.194:33353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 14:49:13.295501 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9651
I20260925 14:49:13.301283 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9284
2026-09-25T14:49:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1902 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T14:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:13Z Disabled control of system clock
I20260925 14:49:13.330860 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42669
--webserver_interface=127.21.205.254
--webserver_port=0
--builtin_ntp_servers=127.21.205.212:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:13.416275 9825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:13.416515 9825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:13.416538 9825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:13.417845 9825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:13.417898 9825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:13.417912 9825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:13.417927 9825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:13.417938 9825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:13.419468 9825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42669
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:13.419809 9825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:13.420162 9825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:13.425318 9833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.425310 9831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.425310 9830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.425542 9825 server_base.cc:1034] running on GCE node
I20260925 14:49:13.425803 9825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:13.426143 9825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:13.427316 9825 hybrid_clock.cc:648] HybridClock initialized: now 1790347753427296 us; error 44 us; skew 500 ppm
I20260925 14:49:13.428665 9825 webserver.cc:460] Webserver started at http://127.21.205.254:44391/ using document root <none> and password file <none>
I20260925 14:49:13.428956 9825 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:13.429005 9825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:13.429198 9825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:13.430337 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "35e84b207a1842bdba3672fef6d415df"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2d397e20942ce2d30f38bf63246e9c25"
server_key_iv: "03732291a79b20b47311257fa02793fb"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.430720 9825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "35e84b207a1842bdba3672fef6d415df"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2d397e20942ce2d30f38bf63246e9c25"
server_key_iv: "03732291a79b20b47311257fa02793fb"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.432174 9825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:13.432983 9839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.433145 9825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:13.433239 9825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "35e84b207a1842bdba3672fef6d415df"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "2d397e20942ce2d30f38bf63246e9c25"
server_key_iv: "03732291a79b20b47311257fa02793fb"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.433303 9825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:13.443348 9825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:13.443660 9825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:13.443845 9825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:13.447718 9825 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42669
I20260925 14:49:13.447837 9892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42669 every 8 connection(s)
I20260925 14:49:13.448215 9825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:13.449303 9893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.451671 9893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: Bootstrap starting.
I20260925 14:49:13.452344 9893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.452672 9893 log.cc:824] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.453400 9893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: No bootstrap required, opened a new log
I20260925 14:49:13.455099 9893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } }
I20260925 14:49:13.455286 9893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.455334 9893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35e84b207a1842bdba3672fef6d415df, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.455497 9893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } }
I20260925 14:49:13.455581 9893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:13.455634 9893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:13.455686 9893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:13.455950 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9825
I20260925 14:49:13.456066 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:13.456234 22327 external_mini_cluster.cc:1428] Setting key 0713540abe06c8f9251295490e44b60f
I20260925 14:49:13.456483 9893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } }
I20260925 14:49:13.456624 9893 leader_election.cc:302] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35e84b207a1842bdba3672fef6d415df; no voters:
I20260925 14:49:13.456866 9893 leader_election.cc:288] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:13.456902 9897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:13.457074 9897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [term 1 LEADER]: Becoming Leader. State: Replica: 35e84b207a1842bdba3672fef6d415df, State: Running, Role: LEADER
I20260925 14:49:13.457224 9897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } }
I20260925 14:49:13.457223 9893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:13.458413 9900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35e84b207a1842bdba3672fef6d415df. Latest consensus state: current_term: 1 leader_uuid: "35e84b207a1842bdba3672fef6d415df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } } }
I20260925 14:49:13.458521 9900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:13.459026 9912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:13.459045 9899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35e84b207a1842bdba3672fef6d415df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35e84b207a1842bdba3672fef6d415df" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42669 } } }
I20260925 14:49:13.459772 9899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:13.460199 9912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:13.461988 9912 catalog_manager.cc:1347] Generated new cluster ID: b91146ac61f54916846af34dc31e4e67
I20260925 14:49:13.462044 9912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:13.462778 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.456775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.21.205.254:42669)
0925 14:49:13.456996 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.457007 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.457398 (+ 391us) server_negotiation.cc:407] Connection header received
0925 14:49:13.457608 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.457615 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.457673 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.457839 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.458028 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.459508 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.460619 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.461157 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.461330 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.462046 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.462050 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.462050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:13.462259 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.462299 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:13.462315 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.462378 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.462485 (+ 107us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.462565 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260925 14:49:13.484179 9912 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:13.484965 9912 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:13.491588 9912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: Generated new TSK 0
I20260925 14:49:13.491873 9912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:13.493723 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.193:0
--local_ip_for_outbound_sockets=127.21.205.193
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--builtin_ntp_servers=127.21.205.212:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 14:49:13.580070 9917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:13.580307 9917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:13.580348 9917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:13.581588 9917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:13.581657 9917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:13.581701 9917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.193
I20260925 14:49:13.583153 9917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.205.193
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.193
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:13.583432 9917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:13.583765 9917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:13.588961 9923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.588979 9922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.589115 9917 server_base.cc:1034] running on GCE node
W20260925 14:49:13.589015 9925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.589491 9917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:13.589779 9917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:13.590924 9917 hybrid_clock.cc:648] HybridClock initialized: now 1790347753590905 us; error 41 us; skew 500 ppm
I20260925 14:49:13.592264 9917 webserver.cc:460] Webserver started at http://127.21.205.193:33241/ using document root <none> and password file <none>
I20260925 14:49:13.592564 9917 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:13.592613 9917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:13.592788 9917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:13.593916 9917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data/instance:
uuid: "f84be9c9b1c74700aa55f0277650a403"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "18ff60d8ddcfc041e55970fe08dd3d42"
server_key_iv: "b73c45b2920f5edbe51fe586b791005b"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.594306 9917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance:
uuid: "f84be9c9b1c74700aa55f0277650a403"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "18ff60d8ddcfc041e55970fe08dd3d42"
server_key_iv: "b73c45b2920f5edbe51fe586b791005b"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.595679 9917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:13.596505 9931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.596680 9917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:13.596786 9917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal
uuid: "f84be9c9b1c74700aa55f0277650a403"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "18ff60d8ddcfc041e55970fe08dd3d42"
server_key_iv: "b73c45b2920f5edbe51fe586b791005b"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.596875 9917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:13.606787 9917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:13.607143 9917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:13.607308 9917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:13.607564 9917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:13.607956 9917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:13.608011 9917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.608054 9917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:13.608081 9917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.613921 9917 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.193:37339
I20260925 14:49:13.613999 10044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.193:37339 every 8 connection(s)
I20260925 14:49:13.614364 9917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/data/info.pb
I20260925 14:49:13.619123 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 9917
I20260925 14:49:13.619259 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-0/wal/instance
I20260925 14:49:13.619480 22327 external_mini_cluster.cc:1428] Setting key 32d54af2f7e5ea6bcf735ad422f71768
I20260925 14:49:13.620189 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.615146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.193:35715 (local address 127.21.205.254:42669)
0925 14:49:13.615216 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.615218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.615481 (+ 263us) server_negotiation.cc:407] Connection header received
0925 14:49:13.616111 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.616112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.616127 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.616191 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.616921 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.617454 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.618523 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.618738 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.618826 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.619866 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.619867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.619868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:13.619874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.619893 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:13.619898 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.619966 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.620070 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.620107 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 14:49:13.620435 10047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.615138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42669 (local address 127.21.205.193:35715)
0925 14:49:13.615359 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.615394 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.616001 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.616273 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.616276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.616294 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:13.616819 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.616823 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.617560 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.617561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.618416 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.618418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.618840 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.618841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.618980 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.619641 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:13.619641 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:13.619695 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:13.619728 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:13.619982 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:13.619990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:13.619992 (+ 2us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.620066 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.620176 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":107,"threads_started":1}
I20260925 14:49:13.620662 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.194:0
--local_ip_for_outbound_sockets=127.21.205.194
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--builtin_ntp_servers=127.21.205.212:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 14:49:13.621419 10045 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42669
I20260925 14:49:13.621532 10045 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:13.621723 10045 heartbeater.cc:511] Master 127.21.205.254:42669 requested a full tablet report, sending...
I20260925 14:49:13.622278 9856 ts_manager.cc:194] Registered new tserver with Master: f84be9c9b1c74700aa55f0277650a403 (127.21.205.193:37339)
I20260925 14:49:13.622916 9856 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.193:35715
W20260925 14:49:13.704979 10048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:13.705240 10048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:13.705279 10048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:13.706521 10048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:13.706593 10048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:13.706638 10048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.194
I20260925 14:49:13.708194 10048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.205.194
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.194
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:13.708515 10048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:13.708858 10048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:13.714133 10056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.714210 10048 server_base.cc:1034] running on GCE node
W20260925 14:49:13.714196 10053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.714133 10054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.714700 10048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:13.714978 10048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:13.716151 10048 hybrid_clock.cc:648] HybridClock initialized: now 1790347753716142 us; error 39 us; skew 500 ppm
I20260925 14:49:13.717463 10048 webserver.cc:460] Webserver started at http://127.21.205.194:36571/ using document root <none> and password file <none>
I20260925 14:49:13.717769 10048 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:13.717845 10048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:13.718009 10048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:13.719148 10048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data/instance:
uuid: "a7ac381df3a54a16b6246e5365eda368"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "cdfdb7ce2dda490a33ba16f88de0b123"
server_key_iv: "9d9731a1506590c74333acaf0d1846d2"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.719538 10048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance:
uuid: "a7ac381df3a54a16b6246e5365eda368"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "cdfdb7ce2dda490a33ba16f88de0b123"
server_key_iv: "9d9731a1506590c74333acaf0d1846d2"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.720995 10048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:13.721786 10062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.721958 10048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:13.722065 10048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal
uuid: "a7ac381df3a54a16b6246e5365eda368"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "cdfdb7ce2dda490a33ba16f88de0b123"
server_key_iv: "9d9731a1506590c74333acaf0d1846d2"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.722149 10048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:13.738505 10048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:13.738907 10048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:13.739077 10048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:13.739327 10048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:13.739696 10048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:13.739751 10048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.739789 10048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:13.739867 10048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.746651 10048 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.194:43387
I20260925 14:49:13.746747 10175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.194:43387 every 8 connection(s)
I20260925 14:49:13.747238 10048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/data/info.pb
I20260925 14:49:13.756182 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.748985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.194:44581 (local address 127.21.205.254:42669)
0925 14:49:13.749041 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.749044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.749333 (+ 289us) server_negotiation.cc:407] Connection header received
0925 14:49:13.750153 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.750156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.750171 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.750258 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.751666 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.752620 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.754316 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.754637 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.754724 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.755872 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.755873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.755873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:13.755881 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.755900 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:13.755905 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.755955 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.756059 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.756092 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:49:13.756346 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10048
I20260925 14:49:13.756493 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-1/wal/instance
I20260925 14:49:13.756701 10178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.748992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42669 (local address 127.21.205.194:44581)
0925 14:49:13.749196 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.749242 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.750018 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.750449 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.750479 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.750490 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:13.751546 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.751554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.752718 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.752719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.754169 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.754173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.754810 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.754811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.754994 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.755660 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:13.755660 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:13.755693 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:13.755721 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:13.755978 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:13.755983 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:13.755987 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.756057 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.756173 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":87,"threads_started":1}
I20260925 14:49:13.756717 22327 external_mini_cluster.cc:1428] Setting key e7d79de407f0632019903cd2a7ca9b09
I20260925 14:49:13.757720 10176 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42669
I20260925 14:49:13.757838 10176 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:13.758001 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.205.195:0
--local_ip_for_outbound_sockets=127.21.205.195
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--builtin_ntp_servers=127.21.205.212:34463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 14:49:13.758064 10176 heartbeater.cc:511] Master 127.21.205.254:42669 requested a full tablet report, sending...
I20260925 14:49:13.758467 9855 ts_manager.cc:194] Registered new tserver with Master: a7ac381df3a54a16b6246e5365eda368 (127.21.205.194:43387)
I20260925 14:49:13.759006 9855 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.194:44581
W20260925 14:49:13.843513 10179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:13.843751 10179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:13.843842 10179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:13.845077 10179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:13.845124 10179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:13.845150 10179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.205.195
I20260925 14:49:13.846446 10179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.205.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.205.195
--webserver_port=0
--tserver_master_addrs=127.21.205.254:42669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.205.195
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:13.846663 10179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:13.846936 10179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:13.852221 10187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.852234 10185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:13.852308 10184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:13.852581 10179 server_base.cc:1034] running on GCE node
I20260925 14:49:13.852905 10179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:13.853168 10179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:13.854341 10179 hybrid_clock.cc:648] HybridClock initialized: now 1790347753854320 us; error 38 us; skew 500 ppm
I20260925 14:49:13.855669 10179 webserver.cc:460] Webserver started at http://127.21.205.195:36145/ using document root <none> and password file <none>
I20260925 14:49:13.855978 10179 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:13.856052 10179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:13.856210 10179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:13.857479 10179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data/instance:
uuid: "03e3fef4067d4e26b342d655f30a2533"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "a850e7280b63800d49e328084412ea6e"
server_key_iv: "7200022ffe3f9d78f2dfef5a4a8c235c"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.857908 10179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance:
uuid: "03e3fef4067d4e26b342d655f30a2533"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "a850e7280b63800d49e328084412ea6e"
server_key_iv: "7200022ffe3f9d78f2dfef5a4a8c235c"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.859340 10179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:13.860211 10193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.860416 10179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260925 14:49:13.860524 10179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal
uuid: "03e3fef4067d4e26b342d655f30a2533"
format_stamp: "Formatted at 2026-09-25 14:49:13 on dist-test-slave-d7lw"
server_key: "a850e7280b63800d49e328084412ea6e"
server_key_iv: "7200022ffe3f9d78f2dfef5a4a8c235c"
server_key_version: "encryptionkey@0"
I20260925 14:49:13.860610 10179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:13.871068 10179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:13.871443 10179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:13.871636 10179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:13.871963 10179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 14:49:13.872356 10179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 14:49:13.872411 10179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.872453 10179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 14:49:13.872480 10179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:13.878432 10179 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.195:33495
I20260925 14:49:13.878857 10179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/data/info.pb
I20260925 14:49:13.879192 10306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.195:33495 every 8 connection(s)
I20260925 14:49:13.883677 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10179
I20260925 14:49:13.883860 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal/instance
I20260925 14:49:13.884092 22327 external_mini_cluster.cc:1428] Setting key 827acd022149aa2763c902226e38c044
I20260925 14:49:13.884763 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.879719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:40383 (local address 127.21.205.254:42669)
0925 14:49:13.879771 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.879774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.880089 (+ 315us) server_negotiation.cc:407] Connection header received
0925 14:49:13.880667 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.880669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.880684 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.880731 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.881475 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.882011 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.883066 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.883281 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.883363 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.884415 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.884417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.884418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:13.884424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.884442 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:13.884446 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.884502 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.884629 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.884667 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:13.885056 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.879714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42669 (local address 127.21.205.195:40383)
0925 14:49:13.879959 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.879996 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.880544 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.880808 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.880812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.880823 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:13.881369 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.881373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.882122 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.882123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.882965 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.882967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.883378 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.883379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.883528 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.884242 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 14:49:13.884243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:13.884293 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 14:49:13.884322 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:13.884507 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:13.884531 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:13.884536 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.884626 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.884788 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":127,"threads_started":1}
I20260925 14:49:13.885530 10307 heartbeater.cc:347] Connected to a master server at 127.21.205.254:42669
I20260925 14:49:13.885632 10307 heartbeater.cc:464] Registering TS with master...
I20260925 14:49:13.885797 10307 heartbeater.cc:511] Master 127.21.205.254:42669 requested a full tablet report, sending...
I20260925 14:49:13.886133 9855 ts_manager.cc:194] Registered new tserver with Master: 03e3fef4067d4e26b342d655f30a2533 (127.21.205.195:33495)
I20260925 14:49:13.886493 9855 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.205.195:40383
I20260925 14:49:13.896301 22327 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 14:49:13.901104 9898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.897722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.21.205.254:42669)
0925 14:49:13.897815 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.897818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.897917 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:49:13.898011 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.898012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.898028 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.898083 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:13.898261 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.898765 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.899598 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.899846 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.899929 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.900735 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:13.900736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:13.900737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:13.900743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:13.900759 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:13.900764 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:13.900854 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.900927 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.900960 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 14:49:13.902886 9855 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43828:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 14:49:13.904650 9855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 14:49:13.911151 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.195:33495 (local address 127.0.0.1:36500)
0925 14:49:13.907443 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.907456 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.907507 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.907906 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.907907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.907910 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.907963 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.907965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.908897 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.908898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.909887 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.909890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.910830 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.910831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.910914 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.910928 (+ 14us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.911006 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.911058 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260925 14:49:13.911166 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.21.205.195:33495)
0925 14:49:13.907642 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.907647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.907658 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:13.907702 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.907705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.907723 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.907905 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.908032 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.908801 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.909993 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.910734 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.910823 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.910924 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.911019 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.911068 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":98,"threads_started":1}
I20260925 14:49:13.911664 10315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43387 (local address 127.0.0.1:34892)
0925 14:49:13.907203 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.907253 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.907335 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.907739 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.907743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.907753 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.907855 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.907858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.908769 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.908770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.910432 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.910433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.911453 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.911454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.911512 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.911516 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.911557 (+ 41us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.911595 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":25,"thread_start_us":134,"threads_started":1}
I20260925 14:49:13.911716 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.21.205.194:43387)
0925 14:49:13.907446 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.907451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.907465 (+ 14us) server_negotiation.cc:407] Connection header received
0925 14:49:13.907508 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.907510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.907525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.907648 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.907943 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.908668 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.910536 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.911364 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.911454 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.911556 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.911584 (+ 28us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.911627 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
I20260925 14:49:13.912220 10241 tablet_service.cc:1467] Processing CreateTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c (DEFAULT_TABLE table=test-table [id=f7f84dc0f2dd420a95420a883754a69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.912496 10110 tablet_service.cc:1467] Processing CreateTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c (DEFAULT_TABLE table=test-table [id=f7f84dc0f2dd420a95420a883754a69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.912684 10241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d0ae54f0d345709f3de4ccfd42c53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.912838 10110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d0ae54f0d345709f3de4ccfd42c53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.913333 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37339 (local address 127.0.0.1:36798)
0925 14:49:13.908823 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:13.908833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:13.908884 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:13.909102 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:13.909104 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:13.909106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:13.909146 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.909147 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.910382 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.910383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.912095 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:13.912098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.913043 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.913044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:13.913124 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.913130 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 14:49:13.913207 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 14:49:13.913258 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1527,"thread_start_us":85,"threads_started":1}
I20260925 14:49:13.913348 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:13.907554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36798 (local address 127.21.205.193:37339)
0925 14:49:13.907804 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:13.907808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:13.908945 (+ 1137us) server_negotiation.cc:407] Connection header received
0925 14:49:13.908993 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:13.908995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:13.909010 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:13.909084 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:13.909212 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.910297 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.912193 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:13.912959 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:13.913039 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:13.913121 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:13.913222 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 14:49:13.913265 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
I20260925 14:49:13.914419 9979 tablet_service.cc:1467] Processing CreateTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c (DEFAULT_TABLE table=test-table [id=f7f84dc0f2dd420a95420a883754a69b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 14:49:13.914793 9979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96d0ae54f0d345709f3de4ccfd42c53c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:13.916260 10321 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Bootstrap starting.
I20260925 14:49:13.916985 10321 tablet_bootstrap.cc:654] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.917335 10321 log.cc:824] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.918080 10324 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: Bootstrap starting.
I20260925 14:49:13.918207 10322 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Bootstrap starting.
I20260925 14:49:13.918195 10321 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: No bootstrap required, opened a new log
I20260925 14:49:13.918501 10321 ts_tablet_manager.cc:1397] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260925 14:49:13.918757 10324 tablet_bootstrap.cc:654] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.919116 10324 log.cc:824] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.920039 10322 tablet_bootstrap.cc:654] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:13.920066 10324 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: No bootstrap required, opened a new log
I20260925 14:49:13.920125 10324 ts_tablet_manager.cc:1397] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260925 14:49:13.920373 10322 log.cc:824] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:13.921228 10322 tablet_bootstrap.cc:492] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: No bootstrap required, opened a new log
I20260925 14:49:13.921301 10322 ts_tablet_manager.cc:1397] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260925 14:49:13.921473 10321 raft_consensus.cc:348] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.921657 10321 raft_consensus.cc:374] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.921706 10321 raft_consensus.cc:729] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03e3fef4067d4e26b342d655f30a2533, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.921830 10321 consensus_queue.cc:260] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.922130 10321 ts_tablet_manager.cc:1428] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260925 14:49:13.922277 10307 heartbeater.cc:503] Master 127.21.205.254:42669 was elected leader, sending a full tablet report...
I20260925 14:49:13.922786 10324 raft_consensus.cc:348] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.922953 10324 raft_consensus.cc:374] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.922986 10324 raft_consensus.cc:729] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f84be9c9b1c74700aa55f0277650a403, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.923106 10324 consensus_queue.cc:260] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.923369 10324 ts_tablet_manager.cc:1428] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260925 14:49:13.923535 10045 heartbeater.cc:503] Master 127.21.205.254:42669 was elected leader, sending a full tablet report...
I20260925 14:49:13.924021 10322 raft_consensus.cc:348] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.924194 10322 raft_consensus.cc:374] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:13.924237 10322 raft_consensus.cc:729] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7ac381df3a54a16b6246e5365eda368, State: Initialized, Role: FOLLOWER
I20260925 14:49:13.924386 10322 consensus_queue.cc:260] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:13.924686 10322 ts_tablet_manager.cc:1428] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260925 14:49:13.924755 10176 heartbeater.cc:503] Master 127.21.205.254:42669 was elected leader, sending a full tablet report...
W20260925 14:49:13.999850 10177 tablet.cc:2369] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 14:49:14.005419 10327 raft_consensus.cc:482] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 14:49:14.005548 10327 raft_consensus.cc:504] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:14.006027 10327 leader_election.cc:288] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7ac381df3a54a16b6246e5365eda368 (127.21.205.194:43387), f84be9c9b1c74700aa55f0277650a403 (127.21.205.193:37339)
I20260925 14:49:14.010241 10320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.006339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:40151 (local address 127.21.205.193:37339)
0925 14:49:14.006392 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:14.006395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:14.006409 (+ 14us) server_negotiation.cc:407] Connection header received
0925 14:49:14.006446 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:14.006447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:14.006458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:14.006519 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:14.006766 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.007557 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009183 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.009781 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009870 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:14.009879 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:14.010103 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 14:49:14.010163 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:14.010241 10318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.006353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.205.195:49349 (local address 127.21.205.194:43387)
0925 14:49:14.006402 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:14.006404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:14.006590 (+ 186us) server_negotiation.cc:407] Connection header received
0925 14:49:14.006618 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:14.006619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:14.006631 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:14.006691 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 14:49:14.006860 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.007554 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009183 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.009787 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009870 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:14.009879 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:14.010098 (+ 219us) server_negotiation.cc:299] Negotiation successful
0925 14:49:14.010150 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:14.010226 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.006229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37339 (local address 127.21.205.195:40151)
0925 14:49:14.006299 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:14.006313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:14.006384 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:14.006550 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:14.006551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:14.006553 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:14.006664 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:14.006666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.007653 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.007654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:14.009044 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:14.009048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.009889 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:14.009980 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:14.009988 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 14:49:14.010053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:49:14.010115 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":80}
I20260925 14:49:14.010237 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.006231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43387 (local address 127.21.205.195:49349)
0925 14:49:14.006485 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:14.006493 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:14.006568 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:14.006713 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:14.006714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:14.006716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 14:49:14.006778 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:14.006779 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.007653 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.007654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:14.009044 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:14.009048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.009889 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.009891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:14.009980 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:14.009988 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 14:49:14.010053 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 14:49:14.010115 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":183,"thread_start_us":94,"threads_started":1}
I20260925 14:49:14.010684 10130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d0ae54f0d345709f3de4ccfd42c53c" candidate_uuid: "03e3fef4067d4e26b342d655f30a2533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ac381df3a54a16b6246e5365eda368" is_pre_election: true
I20260925 14:49:14.010679 9999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d0ae54f0d345709f3de4ccfd42c53c" candidate_uuid: "03e3fef4067d4e26b342d655f30a2533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84be9c9b1c74700aa55f0277650a403" is_pre_election: true
I20260925 14:49:14.010866 10130 raft_consensus.cc:2454] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e3fef4067d4e26b342d655f30a2533 in term 0.
I20260925 14:49:14.010876 9999 raft_consensus.cc:2454] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 03e3fef4067d4e26b342d655f30a2533 in term 0.
I20260925 14:49:14.011103 10194 leader_election.cc:302] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e3fef4067d4e26b342d655f30a2533, a7ac381df3a54a16b6246e5365eda368; no voters:
I20260925 14:49:14.011245 10327 raft_consensus.cc:2781] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 14:49:14.011304 10327 raft_consensus.cc:482] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 14:49:14.011336 10327 raft_consensus.cc:3037] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:14.012162 10327 raft_consensus.cc:504] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:14.012341 10327 leader_election.cc:288] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [CANDIDATE]: Term 1 election: Requested vote from peers a7ac381df3a54a16b6246e5365eda368 (127.21.205.194:43387), f84be9c9b1c74700aa55f0277650a403 (127.21.205.193:37339)
I20260925 14:49:14.012552 10130 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d0ae54f0d345709f3de4ccfd42c53c" candidate_uuid: "03e3fef4067d4e26b342d655f30a2533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7ac381df3a54a16b6246e5365eda368"
I20260925 14:49:14.012557 9999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96d0ae54f0d345709f3de4ccfd42c53c" candidate_uuid: "03e3fef4067d4e26b342d655f30a2533" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f84be9c9b1c74700aa55f0277650a403"
I20260925 14:49:14.012669 10130 raft_consensus.cc:3037] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:14.012674 9999 raft_consensus.cc:3037] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:14.013403 10130 raft_consensus.cc:2454] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e3fef4067d4e26b342d655f30a2533 in term 1.
I20260925 14:49:14.013469 9999 raft_consensus.cc:2454] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 03e3fef4067d4e26b342d655f30a2533 in term 1.
I20260925 14:49:14.013573 10194 leader_election.cc:302] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03e3fef4067d4e26b342d655f30a2533, a7ac381df3a54a16b6246e5365eda368; no voters:
I20260925 14:49:14.013674 10327 raft_consensus.cc:2781] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:14.013828 10327 raft_consensus.cc:686] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 1 LEADER]: Becoming Leader. State: Replica: 03e3fef4067d4e26b342d655f30a2533, State: Running, Role: LEADER
I20260925 14:49:14.013940 10327 consensus_queue.cc:237] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } }
I20260925 14:49:14.014871 9856 catalog_manager.cc:5495] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 reported cstate change: term changed from 0 to 1, leader changed from <none> to 03e3fef4067d4e26b342d655f30a2533 (127.21.205.195). New cstate: current_term: 1 leader_uuid: "03e3fef4067d4e26b342d655f30a2533" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03e3fef4067d4e26b342d655f30a2533" member_type: VOTER last_known_addr { host: "127.21.205.195" port: 33495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 } health_report { overall_health: UNKNOWN } } }
I20260925 14:49:14.027598 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.024990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36516 (local address 127.21.205.195:33495)
0925 14:49:14.025061 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:14.025064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:14.025074 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:49:14.025127 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:14.025128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:14.025151 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:14.025232 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:14.025407 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.025922 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.026718 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:14.026920 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:14.027009 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:14.027207 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:14.027323 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:14.027416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:14.027455 (+ 39us) server_negotiation.cc:299] Negotiation successful
0925 14:49:14.027477 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:49:14.029614 9999 raft_consensus.cc:1261] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 1 FOLLOWER]: Refusing update from remote peer 03e3fef4067d4e26b342d655f30a2533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:14.029614 10130 raft_consensus.cc:1261] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 1 FOLLOWER]: Refusing update from remote peer 03e3fef4067d4e26b342d655f30a2533: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:14.030000 10331 consensus_queue.cc:1035] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f84be9c9b1c74700aa55f0277650a403" member_type: VOTER last_known_addr { host: "127.21.205.193" port: 37339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:14.030140 10327 consensus_queue.cc:1035] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7ac381df3a54a16b6246e5365eda368" member_type: VOTER last_known_addr { host: "127.21.205.194" port: 43387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:14.035037 9856 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:14.035158 9856 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43828:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 14:49:14.036015 9856 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35e84b207a1842bdba3672fef6d415df: Sending DeleteTablet for 3 replicas of tablet 96d0ae54f0d345709f3de4ccfd42c53c
I20260925 14:49:14.036362 10241 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:14 UTC) from {username='slave'} at 127.0.0.1:36500
I20260925 14:49:14.036419 9979 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:14 UTC) from {username='slave'} at 127.0.0.1:36798
I20260925 14:49:14.036417 10110 tablet_service.cc:1514] Processing DeleteTablet for tablet 96d0ae54f0d345709f3de4ccfd42c53c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 14:49:14 UTC) from {username='slave'} at 127.0.0.1:34892
I20260925 14:49:14.036623 10345 tablet_replica.cc:330] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403: stopping tablet replica
I20260925 14:49:14.036726 10345 raft_consensus.cc:2229] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:14.036846 10345 raft_consensus.cc:2258] T 96d0ae54f0d345709f3de4ccfd42c53c P f84be9c9b1c74700aa55f0277650a403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:14.036875 10344 tablet_replica.cc:330] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: stopping tablet replica
I20260925 14:49:14.037024 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9917
I20260925 14:49:14.037045 10344 raft_consensus.cc:2229] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 1 LEADER]: Raft consensus shutting down.
I20260925 14:49:14.037236 10344 raft_consensus.cc:2258] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:14.037801 10344 ts_tablet_manager.cc:1905] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 14:49:14.039441 10344 ts_tablet_manager.cc:1918] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 14:49:14.039546 10344 log.cc:1196] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Deleting WAL directory at /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790347486435597-22327-0/minicluster-data/ts-2/wal/wals/96d0ae54f0d345709f3de4ccfd42c53c
I20260925 14:49:14.039858 10344 ts_tablet_manager.cc:1939] T 96d0ae54f0d345709f3de4ccfd42c53c P 03e3fef4067d4e26b342d655f30a2533: Deleting consensus metadata
I20260925 14:49:14.040297 9843 catalog_manager.cc:4841] TS 03e3fef4067d4e26b342d655f30a2533 (127.21.205.195:33495): tablet 96d0ae54f0d345709f3de4ccfd42c53c (table test-table [id=f7f84dc0f2dd420a95420a883754a69b]) successfully deleted
W20260925 14:49:14.043979 9843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.205.193:37339 (error 108)
I20260925 14:49:14.044170 10346 tablet_replica.cc:330] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368: stopping tablet replica
I20260925 14:49:14.044239 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10048
I20260925 14:49:14.044315 10346 raft_consensus.cc:2229] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 14:49:14.044452 10346 raft_consensus.cc:2258] T 96d0ae54f0d345709f3de4ccfd42c53c P a7ac381df3a54a16b6246e5365eda368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 14:49:14.044972 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.044405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.193:37339 (local address 127.0.0.1:36804)
0925 14:49:14.044872 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:14.044933 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438}
W20260925 14:49:14.046017 9843 catalog_manager.cc:4584] TS f84be9c9b1c74700aa55f0277650a403 (127.21.205.193:37339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96d0ae54f0d345709f3de4ccfd42c53c: Network error: Client connection negotiation failed: client connection to 127.21.205.193:37339: connect: Connection refused (error 111)
I20260925 14:49:14.050112 10319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:14.049969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.194:43387 (local address 127.0.0.1:34900)
0925 14:49:14.050031 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:14.050067 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.194:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:14.050272 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10179
I20260925 14:49:14.055302 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 9825
2026-09-25T14:49:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (752 ms)
[----------] 6 tests from EncryptionPolicyTest (8261 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:14 dist-test-slave-d7lw krb5kdc[10350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:14 dist-test-slave-d7lw krb5kdc[10350](info): set up 2 sockets
Sep 25 14:49:14 dist-test-slave-d7lw krb5kdc[10350](info): commencing operation
krb5kdc: starting...
W20260925 14:49:15.119048 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:15.238662 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36335
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:15.321694 10363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:15.321943 10363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:15.321969 10363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:15.323168 10363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:15.323210 10363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:15.323223 10363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:15.323236 10363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:15.323254 10363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:15.324820 10363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:15.325050 10363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:15.325259 10363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:15.330895 10371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:15.330889 10369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:15.330889 10368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:15.330915 10363 server_base.cc:1034] running on GCE node
I20260925 14:49:15.331404 10363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:15.331730 10363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:15.332932 10363 hybrid_clock.cc:648] HybridClock initialized: now 1790347755332918 us; error 31 us; skew 500 ppm
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347755, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:15.339228 10363 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:15.339705 10363 webserver.cc:460] Webserver started at http://127.0.0.1:43013/ using document root <none> and password file <none>
I20260925 14:49:15.339998 10363 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:15.340050 10363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:15.340147 10363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:15.341257 10363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "670aa07d1f1442d1ae0d92c77a2d6aae"
format_stamp: "Formatted at 2026-09-25 14:49:15 on dist-test-slave-d7lw"
server_key: "aaaeb00773e54caf0d7029c717d736af"
server_key_iv: "a31fd5d08e226a78847287703be9045f"
server_key_version: "encryptionkey@0"
I20260925 14:49:15.341569 10363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "670aa07d1f1442d1ae0d92c77a2d6aae"
format_stamp: "Formatted at 2026-09-25 14:49:15 on dist-test-slave-d7lw"
server_key: "aaaeb00773e54caf0d7029c717d736af"
server_key_iv: "a31fd5d08e226a78847287703be9045f"
server_key_version: "encryptionkey@0"
I20260925 14:49:15.342948 10363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:15.343736 10378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:15.343967 10363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:15.344081 10363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "670aa07d1f1442d1ae0d92c77a2d6aae"
format_stamp: "Formatted at 2026-09-25 14:49:15 on dist-test-slave-d7lw"
server_key: "aaaeb00773e54caf0d7029c717d736af"
server_key_iv: "a31fd5d08e226a78847287703be9045f"
server_key_version: "encryptionkey@0"
I20260925 14:49:15.344175 10363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:15.367844 10363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:15.369876 10363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:15.370055 10363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:15.374601 10363 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36335
I20260925 14:49:15.374670 10430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36335 every 8 connection(s)
I20260925 14:49:15.375061 10363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:15.376160 10431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:15.378602 10431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae: Bootstrap starting.
I20260925 14:49:15.379251 10431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:15.379568 10431 log.cc:824] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:15.380527 10431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae: No bootstrap required, opened a new log
I20260925 14:49:15.382229 10431 raft_consensus.cc:348] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260925 14:49:15.382398 10431 raft_consensus.cc:374] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:15.382421 10431 raft_consensus.cc:729] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670aa07d1f1442d1ae0d92c77a2d6aae, State: Initialized, Role: FOLLOWER
I20260925 14:49:15.382611 10431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260925 14:49:15.382680 10431 raft_consensus.cc:388] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:15.382702 10431 raft_consensus.cc:482] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:15.382769 10431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:15.383422 10431 raft_consensus.cc:504] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260925 14:49:15.383517 10431 leader_election.cc:302] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 670aa07d1f1442d1ae0d92c77a2d6aae; no voters:
I20260925 14:49:15.383756 10431 leader_election.cc:288] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:15.383876 10434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:15.384068 10434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [term 1 LEADER]: Becoming Leader. State: Replica: 670aa07d1f1442d1ae0d92c77a2d6aae, State: Running, Role: LEADER
I20260925 14:49:15.384337 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10363
I20260925 14:49:15.384454 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:15.384644 22327 external_mini_cluster.cc:1428] Setting key 80849a2d59cf6685275a03ed3dfd1c85
I20260925 14:49:15.384091 10431 sys_catalog.cc:564] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:15.384168 10434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } }
I20260925 14:49:15.385512 10436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 670aa07d1f1442d1ae0d92c77a2d6aae. Latest consensus state: current_term: 1 leader_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } } }
I20260925 14:49:15.385617 10436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:15.385830 10435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "670aa07d1f1442d1ae0d92c77a2d6aae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36335 } } }
I20260925 14:49:15.385922 10435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:15.386489 10445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:15.387142 10445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:15.388967 10445 catalog_manager.cc:1347] Generated new cluster ID: 3e3e5420cdcf460b9bb7e0ec9864aada
I20260925 14:49:15.389031 10445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 14:49:15.396311 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:15.385285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48011 (local address 127.0.0.1:36335)
0925 14:49:15.385778 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:15.385788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:15.385821 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:49:15.386147 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:15.386157 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:15.386229 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:15.387114 (+ 885us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:15.387246 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:15.388514 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:15.389511 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:15.389989 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:15.390108 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:15.392669 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:15.392674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:15.392675 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:15.392691 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:15.394529 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:15.394972 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:15.394977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:15.394977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:15.395094 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:15.395335 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:15.395336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:15.395337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:15.395609 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:15.395818 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:15.395962 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 14:49:15.396075 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":74,"threads_started":1}
I20260925 14:49:15.397405 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:15.406417 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:15.398802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37419 (local address 127.0.0.1:36335)
0925 14:49:15.398844 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:15.398846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:15.400070 (+ 1224us) server_negotiation.cc:407] Connection header received
0925 14:49:15.400138 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:15.400139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:15.400150 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:15.400220 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:15.400792 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:15.401321 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:15.402045 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:15.402287 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:15.402362 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:15.403638 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:15.403640 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:15.403640 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:15.403648 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:15.405156 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:15.405554 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:15.405556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:15.405556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:15.405616 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:15.405853 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:15.405854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:15.405855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:15.405980 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:15.406106 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:15.406217 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 14:49:15.406292 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":20}
I20260925 14:49:15.412091 10445 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:15.412708 10445 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:15.418366 10445 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 670aa07d1f1442d1ae0d92c77a2d6aae: Generated new TSK 0
I20260925 14:49:15.418603 10445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:15.422724 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10363
W20260925 14:49:15.427765 10348 connection.cc:535] client connection to 127.0.0.1:36335 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36335): Connection reset by peer (error 104)
2026-09-25T14:49:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10462](info): set up 2 sockets
Sep 25 14:49:15 dist-test-slave-d7lw krb5kdc[10462](info): commencing operation
krb5kdc: starting...
W20260925 14:49:16.466631 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:16 dist-test-slave-d7lw krb5kdc[10462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347756, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:16.579130 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40981
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40981
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:16.661149 10475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:16.661368 10475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:16.661407 10475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:16.662539 10475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:16.662601 10475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:16.662627 10475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:16.662664 10475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:16.662706 10475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:16.664194 10475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40981
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:16.664456 10475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:16.664669 10475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:16.669368 10481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:16.669378 10483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:16.669458 10475 server_base.cc:1034] running on GCE node
W20260925 14:49:16.669406 10480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:16.669711 10475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:16.670015 10475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:16.671200 10475 hybrid_clock.cc:648] HybridClock initialized: now 1790347756671191 us; error 33 us; skew 500 ppm
Sep 25 14:49:16 dist-test-slave-d7lw krb5kdc[10462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347756, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:16.673501 10475 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:16.673885 10475 webserver.cc:460] Webserver started at http://127.0.0.1:41383/ using document root <none> and password file <none>
I20260925 14:49:16.674124 10475 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:16.674196 10475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:16.674340 10475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:16.675304 10475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7"
format_stamp: "Formatted at 2026-09-25 14:49:16 on dist-test-slave-d7lw"
server_key: "a393396bae4f2e6d94e0dc55628aa372"
server_key_iv: "393aa2340df3c592443b3b723d0c7ed9"
server_key_version: "encryptionkey@0"
I20260925 14:49:16.675653 10475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7"
format_stamp: "Formatted at 2026-09-25 14:49:16 on dist-test-slave-d7lw"
server_key: "a393396bae4f2e6d94e0dc55628aa372"
server_key_iv: "393aa2340df3c592443b3b723d0c7ed9"
server_key_version: "encryptionkey@0"
I20260925 14:49:16.676929 10475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:16.677601 10490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:16.677772 10475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:16.677867 10475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7"
format_stamp: "Formatted at 2026-09-25 14:49:16 on dist-test-slave-d7lw"
server_key: "a393396bae4f2e6d94e0dc55628aa372"
server_key_iv: "393aa2340df3c592443b3b723d0c7ed9"
server_key_version: "encryptionkey@0"
I20260925 14:49:16.677953 10475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:16.692579 10475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:16.694329 10475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:16.694485 10475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:16.698402 10475 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40981
I20260925 14:49:16.698437 10544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40981 every 8 connection(s)
I20260925 14:49:16.698772 10475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:16.699653 10545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:16.699759 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10475
I20260925 14:49:16.699916 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:16.700125 22327 external_mini_cluster.cc:1428] Setting key 89b9134184650447becaf67f48a08958
I20260925 14:49:16.702764 10545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7: Bootstrap starting.
I20260925 14:49:16.703460 10545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:16.703739 10545 log.cc:824] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:16.704495 10545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7: No bootstrap required, opened a new log
I20260925 14:49:16.705503 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:16.700647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58253 (local address 127.0.0.1:40981)
0925 14:49:16.700851 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:16.700859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:16.700889 (+ 30us) server_negotiation.cc:407] Connection header received
0925 14:49:16.701311 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:16.701317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:16.701358 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:16.701463 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:16.701673 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:16.702510 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:16.703583 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:16.704127 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:16.704266 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:16.704930 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:16.704934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:16.704935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:16.705093 (+ 158us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:16.705121 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:16.705131 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:16.705232 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:16.705311 (+ 79us) server_negotiation.cc:299] Negotiation successful
0925 14:49:16.705337 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":90,"threads_started":1}
I20260925 14:49:16.706655 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:16.706719 10545 raft_consensus.cc:348] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260925 14:49:16.706909 10545 raft_consensus.cc:374] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:16.706933 10545 raft_consensus.cc:729] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db8cc83af6e14c3c8cb64eca3a6d6ea7, State: Initialized, Role: FOLLOWER
I20260925 14:49:16.707062 10545 consensus_queue.cc:260] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260925 14:49:16.707134 10545 raft_consensus.cc:388] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:16.707155 10545 raft_consensus.cc:482] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:16.707216 10545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:16.707829 10545 raft_consensus.cc:504] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260925 14:49:16.707947 10545 leader_election.cc:302] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db8cc83af6e14c3c8cb64eca3a6d6ea7; no voters:
I20260925 14:49:16.708141 10545 leader_election.cc:288] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:16.708204 10555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:16.708343 10555 raft_consensus.cc:686] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [term 1 LEADER]: Becoming Leader. State: Replica: db8cc83af6e14c3c8cb64eca3a6d6ea7, State: Running, Role: LEADER
I20260925 14:49:16.708374 10545 sys_catalog.cc:564] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:16.708473 10555 consensus_queue.cc:237] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } }
I20260925 14:49:16.709024 10557 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db8cc83af6e14c3c8cb64eca3a6d6ea7. Latest consensus state: current_term: 1 leader_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } } }
I20260925 14:49:16.709139 10557 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:16.709494 10564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:16.709995 10564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:16.709482 10556 sys_catalog.cc:455] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db8cc83af6e14c3c8cb64eca3a6d6ea7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40981 } } }
I20260925 14:49:16.710661 10556 sys_catalog.cc:458] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:16.711884 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:16.708017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49745 (local address 127.0.0.1:40981)
0925 14:49:16.708053 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:16.708055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:16.708531 (+ 476us) server_negotiation.cc:407] Connection header received
0925 14:49:16.708582 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:16.708583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:16.708591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:16.708630 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:16.708836 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:16.709512 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:16.710518 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:16.710801 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:16.710855 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:16.711613 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:16.711614 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:16.711614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:16.711618 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:16.711631 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:16.711635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:16.711706 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:16.711808 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 14:49:16.711819 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:16.711889 10564 catalog_manager.cc:1347] Generated new cluster ID: 87e35a0a796140dea67cde151b77dc0a
I20260925 14:49:16.711962 10564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:16.729141 10564 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:16.729828 10564 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:16.737849 10564 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db8cc83af6e14c3c8cb64eca3a6d6ea7: Generated new TSK 0
I20260925 14:49:16.738014 10564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:16.747637 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10475
W20260925 14:49:16.752717 10460 connection.cc:535] client connection to 127.0.0.1:40981 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40981): Connection reset by peer (error 104)
2026-09-25T14:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:16 dist-test-slave-d7lw krb5kdc[10576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:16 dist-test-slave-d7lw krb5kdc[10576](info): set up 2 sockets
Sep 25 14:49:16 dist-test-slave-d7lw krb5kdc[10576](info): commencing operation
krb5kdc: starting...
W20260925 14:49:17.790712 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:17 dist-test-slave-d7lw krb5kdc[10576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:17.907028 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:17.986425 10589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:17.986620 10589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:17.986642 10589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:17.987676 10589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:17.987712 10589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:17.987727 10589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:17.987739 10589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:17.987751 10589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:17.989194 10589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:17.989367 10589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:17.989527 10589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:49:17.994220 10589 server_base.cc:1034] running on GCE node
W20260925 14:49:17.994158 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:17.994191 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:17.994158 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:17.994587 10589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:17.994902 10589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:17.996060 10589 hybrid_clock.cc:648] HybridClock initialized: now 1790347757996050 us; error 42 us; skew 500 ppm
Sep 25 14:49:17 dist-test-slave-d7lw krb5kdc[10576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347757, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:17.998277 10589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:17.998663 10589 webserver.cc:460] Webserver started at http://127.0.0.1:37345/ using document root <none> and password file <none>
I20260925 14:49:17.998981 10589 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:17.999060 10589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:17.999210 10589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:18.000288 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "4301b3570d8148b5a1b788e1f9f45f83"
format_stamp: "Formatted at 2026-09-25 14:49:17 on dist-test-slave-d7lw"
server_key: "86eb4080a223782e39e00cae21879293"
server_key_iv: "833191b679d4ae04e7ec175811516726"
server_key_version: "encryptionkey@0"
I20260925 14:49:18.000635 10589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "4301b3570d8148b5a1b788e1f9f45f83"
format_stamp: "Formatted at 2026-09-25 14:49:17 on dist-test-slave-d7lw"
server_key: "86eb4080a223782e39e00cae21879293"
server_key_iv: "833191b679d4ae04e7ec175811516726"
server_key_version: "encryptionkey@0"
I20260925 14:49:18.002031 10589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:18.002774 10604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:18.002985 10589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:18.003083 10589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "4301b3570d8148b5a1b788e1f9f45f83"
format_stamp: "Formatted at 2026-09-25 14:49:17 on dist-test-slave-d7lw"
server_key: "86eb4080a223782e39e00cae21879293"
server_key_iv: "833191b679d4ae04e7ec175811516726"
server_key_version: "encryptionkey@0"
I20260925 14:49:18.003163 10589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:18.006393 10589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:18.008090 10589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:18.008248 10589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:18.012079 10589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37713
I20260925 14:49:18.012112 10657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37713 every 8 connection(s)
I20260925 14:49:18.012423 10589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:18.013353 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:18.015389 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83: Bootstrap starting.
I20260925 14:49:18.015951 10658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:18.016189 10658 log.cc:824] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:18.016796 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83: No bootstrap required, opened a new log
I20260925 14:49:18.018261 10658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } }
I20260925 14:49:18.018385 10658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:18.018406 10658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4301b3570d8148b5a1b788e1f9f45f83, State: Initialized, Role: FOLLOWER
I20260925 14:49:18.018530 10658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } }
I20260925 14:49:18.018608 10658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:18.018631 10658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:18.018657 10658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:18.019129 10658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } }
I20260925 14:49:18.019207 10658 leader_election.cc:302] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4301b3570d8148b5a1b788e1f9f45f83; no voters:
I20260925 14:49:18.019326 10658 leader_election.cc:288] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:18.019397 10661 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:18.019559 10661 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [term 1 LEADER]: Becoming Leader. State: Replica: 4301b3570d8148b5a1b788e1f9f45f83, State: Running, Role: LEADER
I20260925 14:49:18.019587 10658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:18.019644 10661 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } }
I20260925 14:49:18.020040 10663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4301b3570d8148b5a1b788e1f9f45f83. Latest consensus state: current_term: 1 leader_uuid: "4301b3570d8148b5a1b788e1f9f45f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } } }
I20260925 14:49:18.020125 10663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:18.020154 10662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4301b3570d8148b5a1b788e1f9f45f83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4301b3570d8148b5a1b788e1f9f45f83" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37713 } } }
I20260925 14:49:18.020210 10662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:18.020622 10669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:18.021150 10669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:18.021519 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10589
I20260925 14:49:18.021642 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:18.021834 22327 external_mini_cluster.cc:1428] Setting key acc16aaa8809520413ca26840badb8b9
I20260925 14:49:18.022619 10669 catalog_manager.cc:1347] Generated new cluster ID: 5cd8b5afcc74473e84eec09afab1680c
I20260925 14:49:18.022708 10669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:18.024385 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:18.022222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43065 (local address 127.0.0.1:37713)
0925 14:49:18.022915 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:18.022924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:18.022971 (+ 47us) server_negotiation.cc:407] Connection header received
0925 14:49:18.023087 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:18.023094 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:18.023139 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:18.023263 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:18.023971 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:18.023974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:18.023974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:18.023978 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:18.024014 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:18.024026 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:18.024084 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:18.024184 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 14:49:18.024219 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":82,"threads_started":1}
I20260925 14:49:18.025195 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:18.027554 10681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:18.026569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40043 (local address 127.0.0.1:37713)
0925 14:49:18.026606 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:18.026608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:18.026688 (+ 80us) server_negotiation.cc:407] Connection header received
0925 14:49:18.026734 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:18.026735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:18.026740 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:18.026791 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:18.027368 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:18.027369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:18.027369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:18.027374 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:18.027385 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:18.027389 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:18.027439 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:18.027491 (+ 52us) server_negotiation.cc:299] Negotiation successful
0925 14:49:18.027501 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:49:18.028887 10669 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:18.029035 10669 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:18.032934 10669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4301b3570d8148b5a1b788e1f9f45f83: Generated new TSK 0
I20260925 14:49:18.033099 10669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:18.036206 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10589
2026-09-25T14:49:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1287 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:18 dist-test-slave-d7lw krb5kdc[10690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:18 dist-test-slave-d7lw krb5kdc[10690](info): set up 2 sockets
Sep 25 14:49:18 dist-test-slave-d7lw krb5kdc[10690](info): commencing operation
krb5kdc: starting...
W20260925 14:49:19.071836 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:19.188230 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:19.268038 10703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:19.268344 10703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:19.268383 10703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:19.269845 10703 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:19.269898 10703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:19.269912 10703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:19.269923 10703 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:19.269968 10703 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:19.271384 10703 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:19.271644 10703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:19.271889 10703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:19.277076 10708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:19.277136 10703 server_base.cc:1034] running on GCE node
W20260925 14:49:19.277066 10711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:19.277077 10709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:19.277550 10703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:19.277803 10703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:19.278947 10703 hybrid_clock.cc:648] HybridClock initialized: now 1790347759278936 us; error 26 us; skew 500 ppm
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347759, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:19.281379 10703 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:19.281857 10703 webserver.cc:460] Webserver started at http://127.0.0.1:33325/ using document root <none> and password file <none>
I20260925 14:49:19.282080 10703 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:19.282125 10703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:19.282286 10703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:19.283376 10703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "707b8d3269ac4025801bbff569465e97"
format_stamp: "Formatted at 2026-09-25 14:49:19 on dist-test-slave-d7lw"
server_key: "d1c89cf511d6d18d0f9dd3c7d107eb24"
server_key_iv: "e325add2fc34d9a23d46bf604285da71"
server_key_version: "encryptionkey@0"
I20260925 14:49:19.283737 10703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "707b8d3269ac4025801bbff569465e97"
format_stamp: "Formatted at 2026-09-25 14:49:19 on dist-test-slave-d7lw"
server_key: "d1c89cf511d6d18d0f9dd3c7d107eb24"
server_key_iv: "e325add2fc34d9a23d46bf604285da71"
server_key_version: "encryptionkey@0"
I20260925 14:49:19.285084 10703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:19.285807 10718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:19.285989 10703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:19.286079 10703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "707b8d3269ac4025801bbff569465e97"
format_stamp: "Formatted at 2026-09-25 14:49:19 on dist-test-slave-d7lw"
server_key: "d1c89cf511d6d18d0f9dd3c7d107eb24"
server_key_iv: "e325add2fc34d9a23d46bf604285da71"
server_key_version: "encryptionkey@0"
I20260925 14:49:19.286155 10703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:19.298892 10703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:19.300738 10703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:19.300889 10703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:19.305017 10703 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40835
I20260925 14:49:19.305047 10770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40835 every 8 connection(s)
I20260925 14:49:19.305393 10703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:19.306255 10771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:19.308493 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: Bootstrap starting.
I20260925 14:49:19.309010 10771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:19.309240 10771 log.cc:824] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:19.309906 10771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: No bootstrap required, opened a new log
I20260925 14:49:19.311400 10771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260925 14:49:19.311532 10771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:19.311553 10771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707b8d3269ac4025801bbff569465e97, State: Initialized, Role: FOLLOWER
I20260925 14:49:19.311650 10771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260925 14:49:19.311728 10771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:19.311800 10771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:19.311856 10771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:19.312239 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10703
I20260925 14:49:19.312374 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:19.312554 22327 external_mini_cluster.cc:1428] Setting key fbe2b6df3bfcfba725b7f9edfb2dc10e
I20260925 14:49:19.312569 10771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260925 14:49:19.312667 10771 leader_election.cc:302] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707b8d3269ac4025801bbff569465e97; no voters:
I20260925 14:49:19.312829 10771 leader_election.cc:288] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:19.312866 10774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:19.313066 10774 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [term 1 LEADER]: Becoming Leader. State: Replica: 707b8d3269ac4025801bbff569465e97, State: Running, Role: LEADER
I20260925 14:49:19.313261 10774 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } }
I20260925 14:49:19.313066 10771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:19.315209 10775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707b8d3269ac4025801bbff569465e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } } }
I20260925 14:49:19.315425 10775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [sys.catalog]: This master's current role is: LEADER
W20260925 14:49:19.315465 10791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:19.315506 10791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:19.315644 10777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707b8d3269ac4025801bbff569465e97. Latest consensus state: current_term: 1 leader_uuid: "707b8d3269ac4025801bbff569465e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707b8d3269ac4025801bbff569465e97" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40835 } } }
I20260925 14:49:19.315729 10777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:19.315923 10793 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:19.316089 10793 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:19.317628 10793 catalog_manager.cc:1347] Generated new cluster ID: f7efa061fd484d06bfc960cab2e01140
I20260925 14:49:19.317687 10793 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10690](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 14:49:19.323467 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:19.313184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43257 (local address 127.0.0.1:40835)
0925 14:49:19.313334 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:19.313344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:19.313379 (+ 35us) server_negotiation.cc:407] Connection header received
0925 14:49:19.313505 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:19.313512 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:19.313590 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:19.313729 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:19.314505 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:19.315834 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:19.316864 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:19.317388 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:19.317520 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:19.319886 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:19.319890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:19.319891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:19.319903 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:19.321953 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:19.322351 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:19.322354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:19.322355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:19.322455 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:19.322675 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:19.322675 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:19.322676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:19.322906 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:19.323075 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:19.323152 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 14:49:19.323303 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260925 14:49:19.324424 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:19.338047 10778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:19.325696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34233 (local address 127.0.0.1:40835)
0925 14:49:19.325733 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:19.325734 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:19.331976 (+ 6242us) server_negotiation.cc:407] Connection header received
0925 14:49:19.332051 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:19.332052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:19.332064 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:19.332112 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:19.332631 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:19.333112 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:19.333809 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:19.334018 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:19.334102 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:19.335226 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:19.335228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:19.335228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:19.335234 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:19.336685 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:19.337028 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:19.337029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:19.337030 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:19.337080 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:19.337284 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:19.337285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:19.337285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:19.337400 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:19.337879 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:19.337917 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 14:49:19.337988 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260925 14:49:19.349538 10793 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:19.350262 10793 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:19.356820 10793 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707b8d3269ac4025801bbff569465e97: Generated new TSK 0
I20260925 14:49:19.357018 10793 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:19.367493 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10703
2026-09-25T14:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10802](info): set up 2 sockets
Sep 25 14:49:19 dist-test-slave-d7lw krb5kdc[10802](info): commencing operation
krb5kdc: starting...
W20260925 14:49:20.398909 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:20 dist-test-slave-d7lw krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:20.512926 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46011
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:20.591528 10815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:20.591717 10815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:20.591745 10815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:20.592881 10815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:20.592923 10815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:20.592937 10815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:20.592949 10815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:20.592962 10815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:20.594324 10815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46011
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:20.594504 10815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:20.594663 10815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:20.599274 10820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:20.599299 10823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:20.599449 10815 server_base.cc:1034] running on GCE node
W20260925 14:49:20.599493 10821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:20.599825 10815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:20.600123 10815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:20.601300 10815 hybrid_clock.cc:648] HybridClock initialized: now 1790347760601280 us; error 36 us; skew 500 ppm
Sep 25 14:49:20 dist-test-slave-d7lw krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347760, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:20.607928 10815 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:20.608392 10815 webserver.cc:460] Webserver started at http://127.0.0.1:46649/ using document root <none> and password file <none>
I20260925 14:49:20.608615 10815 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:20.608681 10815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:20.608847 10815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:20.609874 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4"
format_stamp: "Formatted at 2026-09-25 14:49:20 on dist-test-slave-d7lw"
server_key: "be4f37884c156e6b541ba16fa0119995"
server_key_iv: "1e1c5d8a79df266a3fe9311d698cf5d4"
server_key_version: "encryptionkey@0"
I20260925 14:49:20.610234 10815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4"
format_stamp: "Formatted at 2026-09-25 14:49:20 on dist-test-slave-d7lw"
server_key: "be4f37884c156e6b541ba16fa0119995"
server_key_iv: "1e1c5d8a79df266a3fe9311d698cf5d4"
server_key_version: "encryptionkey@0"
I20260925 14:49:20.611595 10815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:49:20.612458 10831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:20.612627 10815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:20.612725 10815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4"
format_stamp: "Formatted at 2026-09-25 14:49:20 on dist-test-slave-d7lw"
server_key: "be4f37884c156e6b541ba16fa0119995"
server_key_iv: "1e1c5d8a79df266a3fe9311d698cf5d4"
server_key_version: "encryptionkey@0"
I20260925 14:49:20.612823 10815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:20.636380 10815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:20.638648 10815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:20.638820 10815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:20.642858 10815 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46011
I20260925 14:49:20.642887 10883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46011 every 8 connection(s)
I20260925 14:49:20.643275 10815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:20.644142 10884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:20.646312 10884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4: Bootstrap starting.
I20260925 14:49:20.646837 10884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:20.647084 10884 log.cc:824] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:20.647387 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10815
I20260925 14:49:20.647501 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:20.647668 22327 external_mini_cluster.cc:1428] Setting key 94651da2663f44417e318b458a3bb3bf
I20260925 14:49:20.647905 10884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4: No bootstrap required, opened a new log
I20260925 14:49:20.649595 10884 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } }
I20260925 14:49:20.649770 10884 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:20.649832 10884 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0fe3b473444484ba6a1be1b1ab6ed4, State: Initialized, Role: FOLLOWER
I20260925 14:49:20.649966 10884 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } }
I20260925 14:49:20.650066 10884 raft_consensus.cc:388] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:20.650110 10884 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:20.650158 10884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:20.650683 10884 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } }
I20260925 14:49:20.650806 10884 leader_election.cc:302] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf0fe3b473444484ba6a1be1b1ab6ed4; no voters:
I20260925 14:49:20.651000 10884 leader_election.cc:288] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:20.651031 10889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:20.651247 10889 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [term 1 LEADER]: Becoming Leader. State: Replica: cf0fe3b473444484ba6a1be1b1ab6ed4, State: Running, Role: LEADER
I20260925 14:49:20.651309 10884 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:20.651358 10889 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } }
I20260925 14:49:20.651803 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf0fe3b473444484ba6a1be1b1ab6ed4. Latest consensus state: current_term: 1 leader_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } } }
I20260925 14:49:20.651822 10891 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf0fe3b473444484ba6a1be1b1ab6ed4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46011 } } }
I20260925 14:49:20.651922 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:20.651950 10891 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:20.652185 10893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:20.652755 10893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:20.654765 10893 catalog_manager.cc:1347] Generated new cluster ID: 10bb42417d24485d99c1f5cfd638e247
I20260925 14:49:20.654814 10893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:20.655005 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:20.648188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59001 (local address 127.0.0.1:46011)
0925 14:49:20.648383 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:20.648392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:20.648420 (+ 28us) server_negotiation.cc:407] Connection header received
0925 14:49:20.649586 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:20.649593 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:20.649659 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:20.649808 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:20.650021 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:20.651556 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:20.652717 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:20.653297 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:20.653399 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:20.654266 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:20.654270 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:20.654270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:20.654500 (+ 230us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:20.654540 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:20.654554 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:20.654630 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:20.654725 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 14:49:20.654792 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260925 14:49:20.655946 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:20.671399 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:20.657278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39223 (local address 127.0.0.1:46011)
0925 14:49:20.657308 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:20.657310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:20.667977 (+ 10667us) server_negotiation.cc:407] Connection header received
0925 14:49:20.668028 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:20.668029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:20.668041 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:20.668104 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:20.668268 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:20.669033 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:20.670069 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:20.670364 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:20.670425 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:20.671142 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:20.671143 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:20.671143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:20.671149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:20.671164 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:20.671168 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:20.671220 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:20.671295 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 14:49:20.671319 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260925 14:49:20.673048 10893 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:20.673569 10893 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:20.680532 10893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cf0fe3b473444484ba6a1be1b1ab6ed4: Generated new TSK 0
I20260925 14:49:20.680716 10893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:20.683672 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10815
2026-09-25T14:49:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:20 dist-test-slave-d7lw krb5kdc[10915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:20 dist-test-slave-d7lw krb5kdc[10915](info): set up 2 sockets
Sep 25 14:49:20 dist-test-slave-d7lw krb5kdc[10915](info): commencing operation
krb5kdc: starting...
W20260925 14:49:21.711128 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:21 dist-test-slave-d7lw krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:21.826468 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:21.906597 10928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:21.906795 10928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:21.906817 10928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:21.907999 10928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:21.908042 10928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:21.908057 10928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:21.908070 10928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:21.908082 10928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:21.909478 10928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:21.909675 10928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:21.909838 10928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:21.914753 10934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:21.914806 10928 server_base.cc:1034] running on GCE node
W20260925 14:49:21.914780 10933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:21.914793 10936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:21.915218 10928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:21.915530 10928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:21.916689 10928 hybrid_clock.cc:648] HybridClock initialized: now 1790347761916676 us; error 36 us; skew 500 ppm
Sep 25 14:49:21 dist-test-slave-d7lw krb5kdc[10915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347761, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:21.919126 10928 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:21.919551 10928 webserver.cc:460] Webserver started at http://127.0.0.1:42683/ using document root <none> and password file <none>
I20260925 14:49:21.919770 10928 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:21.919884 10928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:21.920034 10928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:21.921116 10928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "0d66bd5965394871a6701757a72339f8"
format_stamp: "Formatted at 2026-09-25 14:49:21 on dist-test-slave-d7lw"
server_key: "9fc42bfcdcfe603aac9f19d121d5c0a9"
server_key_iv: "ebce0df8e5e18c0f410bda860c67d620"
server_key_version: "encryptionkey@0"
I20260925 14:49:21.921480 10928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "0d66bd5965394871a6701757a72339f8"
format_stamp: "Formatted at 2026-09-25 14:49:21 on dist-test-slave-d7lw"
server_key: "9fc42bfcdcfe603aac9f19d121d5c0a9"
server_key_iv: "ebce0df8e5e18c0f410bda860c67d620"
server_key_version: "encryptionkey@0"
I20260925 14:49:21.922838 10928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:21.923592 10943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:21.923766 10928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:21.923892 10928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "0d66bd5965394871a6701757a72339f8"
format_stamp: "Formatted at 2026-09-25 14:49:21 on dist-test-slave-d7lw"
server_key: "9fc42bfcdcfe603aac9f19d121d5c0a9"
server_key_iv: "ebce0df8e5e18c0f410bda860c67d620"
server_key_version: "encryptionkey@0"
I20260925 14:49:21.923975 10928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:21.927193 10928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:21.929450 10928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:21.929602 10928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:21.933542 10928 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33041
I20260925 14:49:21.933576 10996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33041 every 8 connection(s)
I20260925 14:49:21.933899 10928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:21.934839 10997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:21.937080 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8: Bootstrap starting.
I20260925 14:49:21.937618 10997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:21.937851 10997 log.cc:824] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:21.938467 10997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8: No bootstrap required, opened a new log
I20260925 14:49:21.940096 10997 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260925 14:49:21.940228 10997 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:21.940250 10997 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d66bd5965394871a6701757a72339f8, State: Initialized, Role: FOLLOWER
I20260925 14:49:21.940380 10997 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260925 14:49:21.940466 10997 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:21.940500 10997 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:21.940570 10997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:21.940577 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 10928
I20260925 14:49:21.940689 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:21.940874 22327 external_mini_cluster.cc:1428] Setting key b5ee01d6f6d44a1086b533fb0bffea83
I20260925 14:49:21.941256 10997 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260925 14:49:21.941455 10997 leader_election.cc:302] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d66bd5965394871a6701757a72339f8; no voters:
I20260925 14:49:21.941672 10997 leader_election.cc:288] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:21.941695 11002 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:21.941804 11002 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [term 1 LEADER]: Becoming Leader. State: Replica: 0d66bd5965394871a6701757a72339f8, State: Running, Role: LEADER
I20260925 14:49:21.941936 10997 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:21.941943 11002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } }
I20260925 14:49:21.942639 11002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d66bd5965394871a6701757a72339f8. Latest consensus state: current_term: 1 leader_uuid: "0d66bd5965394871a6701757a72339f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260925 14:49:21.942708 11002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:21.942972 11004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d66bd5965394871a6701757a72339f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d66bd5965394871a6701757a72339f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33041 } } }
I20260925 14:49:21.943090 11007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:21.943130 11004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:21.942956 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:21.941342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40491 (local address 127.0.0.1:33041)
0925 14:49:21.941511 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:21.941520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:21.941557 (+ 37us) server_negotiation.cc:407] Connection header received
0925 14:49:21.941670 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:21.941675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:21.941698 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:21.941855 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:21.942484 (+ 629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:21.942488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:21.942488 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:21.942492 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:21.942539 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:21.942550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:21.942622 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:21.942691 (+ 69us) server_negotiation.cc:299] Negotiation successful
0925 14:49:21.942731 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260925 14:49:21.943933 11007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:21.944263 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:21.945647 11007 catalog_manager.cc:1347] Generated new cluster ID: 8678c28dd47047db9e06443a679658e1
I20260925 14:49:21.945690 11007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:21.949234 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:21.948150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36921 (local address 127.0.0.1:33041)
0925 14:49:21.948189 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:21.948191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:21.948304 (+ 113us) server_negotiation.cc:407] Connection header received
0925 14:49:21.948362 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:21.948362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:21.948367 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:21.948415 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:21.949008 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:21.949009 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:21.949010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:21.949013 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:21.949029 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:21.949033 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:21.949087 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:21.949158 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 14:49:21.949171 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:49:21.967643 11007 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:21.967880 11007 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:21.976433 11007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d66bd5965394871a6701757a72339f8: Generated new TSK 0
I20260925 14:49:21.976655 11007 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:21.981427 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 10928
2026-09-25T14:49:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:22 dist-test-slave-d7lw krb5kdc[11028](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:22 dist-test-slave-d7lw krb5kdc[11028](info): set up 2 sockets
Sep 25 14:49:22 dist-test-slave-d7lw krb5kdc[11028](info): commencing operation
krb5kdc: starting...
W20260925 14:49:23.018925 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:23.133023 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:23.212929 11041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:23.213119 11041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:23.213140 11041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:23.214278 11041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:23.214319 11041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:23.214334 11041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:23.214347 11041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:23.214359 11041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:23.215903 11041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:23.216136 11041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:23.216367 11041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:23.221019 11049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:23.221040 11047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:23.221025 11046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:23.221374 11041 server_base.cc:1034] running on GCE node
I20260925 14:49:23.221609 11041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:23.221933 11041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:23.223111 11041 hybrid_clock.cc:648] HybridClock initialized: now 1790347763223073 us; error 35 us; skew 500 ppm
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11028](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347763, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:23.225666 11041 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:23.226116 11041 webserver.cc:460] Webserver started at http://127.0.0.1:34797/ using document root <none> and password file <none>
I20260925 14:49:23.226339 11041 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:23.226409 11041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:23.226560 11041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:23.227612 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "514952cf24d346339686a0fb3da3624e"
format_stamp: "Formatted at 2026-09-25 14:49:23 on dist-test-slave-d7lw"
server_key: "0179c9f835b0e70a232304a380e86840"
server_key_iv: "67da87a2914498de1cf4259055b54d8f"
server_key_version: "encryptionkey@0"
I20260925 14:49:23.228020 11041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "514952cf24d346339686a0fb3da3624e"
format_stamp: "Formatted at 2026-09-25 14:49:23 on dist-test-slave-d7lw"
server_key: "0179c9f835b0e70a232304a380e86840"
server_key_iv: "67da87a2914498de1cf4259055b54d8f"
server_key_version: "encryptionkey@0"
I20260925 14:49:23.229341 11041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:23.230031 11056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:23.230213 11041 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:23.230324 11041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "514952cf24d346339686a0fb3da3624e"
format_stamp: "Formatted at 2026-09-25 14:49:23 on dist-test-slave-d7lw"
server_key: "0179c9f835b0e70a232304a380e86840"
server_key_iv: "67da87a2914498de1cf4259055b54d8f"
server_key_version: "encryptionkey@0"
I20260925 14:49:23.230402 11041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:23.240039 11041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:23.241935 11041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:23.242105 11041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:23.246133 11041 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38653
I20260925 14:49:23.246167 11108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38653 every 8 connection(s)
I20260925 14:49:23.246536 11041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:23.247195 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11041
I20260925 14:49:23.247320 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:23.247539 22327 external_mini_cluster.cc:1428] Setting key 2b53e3d21f9acd2009092e89aac2426a
I20260925 14:49:23.247700 11109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:23.251206 11109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e: Bootstrap starting.
I20260925 14:49:23.251899 11109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:23.252228 11109 log.cc:824] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:23.252969 11109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e: No bootstrap required, opened a new log
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11028](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 14:49:23.254850 11109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } }
I20260925 14:49:23.255018 11109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:23.255062 11109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 514952cf24d346339686a0fb3da3624e, State: Initialized, Role: FOLLOWER
I20260925 14:49:23.255201 11109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } }
I20260925 14:49:23.255298 11109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:23.255343 11109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:23.255640 11109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:23.256444 11109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } }
I20260925 14:49:23.256553 11109 leader_election.cc:302] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 514952cf24d346339686a0fb3da3624e; no voters:
I20260925 14:49:23.256716 11109 leader_election.cc:288] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:23.256780 11115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:23.256968 11115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [term 1 LEADER]: Becoming Leader. State: Replica: 514952cf24d346339686a0fb3da3624e, State: Running, Role: LEADER
I20260925 14:49:23.256994 11109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:23.257092 11115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } }
I20260925 14:49:23.257422 11115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 514952cf24d346339686a0fb3da3624e. Latest consensus state: current_term: 1 leader_uuid: "514952cf24d346339686a0fb3da3624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } } }
I20260925 14:49:23.257499 11115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:23.257651 11116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "514952cf24d346339686a0fb3da3624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "514952cf24d346339686a0fb3da3624e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38653 } } }
I20260925 14:49:23.257730 11116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:23.257977 11122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:23.258539 11122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:23.259380 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:23.248162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:60895 (local address 127.0.0.1:38653)
0925 14:49:23.248368 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:23.248378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:23.248411 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:49:23.248999 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:23.249009 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:23.249076 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:23.249243 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:23.249844 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:23.251218 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:23.252091 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:23.252678 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:23.252788 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:23.255066 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:23.255070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:23.255070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:23.255083 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:23.257119 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:23.257684 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:23.257686 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:23.257687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:23.257808 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:23.258095 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:23.258096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:23.258096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:23.258424 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:23.258984 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:23.259069 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 14:49:23.259167 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":70,"threads_started":1}
I20260925 14:49:23.260322 11122 catalog_manager.cc:1347] Generated new cluster ID: 1a1c4ac37d01422baa53eb0d5fae4013
I20260925 14:49:23.260373 11122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:23.260465 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:23.269719 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:23.261929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:50521 (local address 127.0.0.1:38653)
0925 14:49:23.261958 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:23.261960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:23.262082 (+ 122us) server_negotiation.cc:407] Connection header received
0925 14:49:23.262179 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:23.262180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:23.262190 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:23.262218 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:23.262946 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:23.263710 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:23.264810 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:23.265212 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:23.265266 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:23.266802 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:23.266803 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:23.266803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:23.266810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:23.268378 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:23.268811 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:23.268812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:23.268812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:23.268867 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:23.269097 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:23.269098 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:23.269098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:23.269220 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:23.269312 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:23.269559 (+ 247us) server_negotiation.cc:299] Negotiation successful
0925 14:49:23.269624 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260925 14:49:23.278857 11122 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:23.279623 11122 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:23.286405 11122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 514952cf24d346339686a0fb3da3624e: Generated new TSK 0
I20260925 14:49:23.286597 11122 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:23.290374 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11041
W20260925 14:49:23.295639 11026 connection.cc:535] client connection to 127.0.0.1:38653 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38653): Connection reset by peer (error 104)
2026-09-25T14:49:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11141](info): set up 2 sockets
Sep 25 14:49:23 dist-test-slave-d7lw krb5kdc[11141](info): commencing operation
krb5kdc: starting...
W20260925 14:49:24.338799 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:24 dist-test-slave-d7lw krb5kdc[11141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:24.462741 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:24.542261 11154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:24.542459 11154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:24.542481 11154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:24.543556 11154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:24.543591 11154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:24.543605 11154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:24.543617 11154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:24.543629 11154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:24.545023 11154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:24.545208 11154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:24.545464 11154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:24.550678 11162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:24.550696 11160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:24.550699 11154 server_base.cc:1034] running on GCE node
W20260925 14:49:24.550674 11159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:24.551105 11154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:24.551401 11154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:24.552561 11154 hybrid_clock.cc:648] HybridClock initialized: now 1790347764552553 us; error 38 us; skew 500 ppm
Sep 25 14:49:24 dist-test-slave-d7lw krb5kdc[11141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347764, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:24.554917 11154 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:24.555400 11154 webserver.cc:460] Webserver started at http://127.0.0.1:39835/ using document root <none> and password file <none>
I20260925 14:49:24.555637 11154 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:24.555708 11154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:24.555887 11154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:24.556903 11154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "248c97d99802445184efa4a8ccf309fc"
format_stamp: "Formatted at 2026-09-25 14:49:24 on dist-test-slave-d7lw"
server_key: "726e77e4d91e3a8f5825366f572f8c63"
server_key_iv: "bce18c99125971acf3d2e1dfc2407602"
server_key_version: "encryptionkey@0"
I20260925 14:49:24.557268 11154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "248c97d99802445184efa4a8ccf309fc"
format_stamp: "Formatted at 2026-09-25 14:49:24 on dist-test-slave-d7lw"
server_key: "726e77e4d91e3a8f5825366f572f8c63"
server_key_iv: "bce18c99125971acf3d2e1dfc2407602"
server_key_version: "encryptionkey@0"
I20260925 14:49:24.558564 11154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:24.559294 11169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:24.559463 11154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:24.559561 11154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "248c97d99802445184efa4a8ccf309fc"
format_stamp: "Formatted at 2026-09-25 14:49:24 on dist-test-slave-d7lw"
server_key: "726e77e4d91e3a8f5825366f572f8c63"
server_key_iv: "bce18c99125971acf3d2e1dfc2407602"
server_key_version: "encryptionkey@0"
I20260925 14:49:24.559645 11154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:24.591657 11154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:24.593622 11154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:24.593793 11154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:24.597939 11154 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45019
I20260925 14:49:24.598023 11221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45019 every 8 connection(s)
I20260925 14:49:24.598382 11154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:24.599242 11222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:24.601636 11222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc: Bootstrap starting.
I20260925 14:49:24.602238 11222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:24.602533 11222 log.cc:824] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:24.603430 11222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc: No bootstrap required, opened a new log
I20260925 14:49:24.605244 11222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260925 14:49:24.605397 11222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:24.605423 11222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 248c97d99802445184efa4a8ccf309fc, State: Initialized, Role: FOLLOWER
I20260925 14:49:24.605623 11222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260925 14:49:24.605696 11222 raft_consensus.cc:388] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:24.605741 11222 raft_consensus.cc:482] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:24.605799 11222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:24.606356 11222 raft_consensus.cc:504] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260925 14:49:24.606472 11222 leader_election.cc:302] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 248c97d99802445184efa4a8ccf309fc; no voters:
I20260925 14:49:24.606673 11222 leader_election.cc:288] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:24.606693 11225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:24.606850 11225 raft_consensus.cc:686] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [term 1 LEADER]: Becoming Leader. State: Replica: 248c97d99802445184efa4a8ccf309fc, State: Running, Role: LEADER
I20260925 14:49:24.606945 11222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:24.607008 11225 consensus_queue.cc:237] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } }
I20260925 14:49:24.607407 11226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "248c97d99802445184efa4a8ccf309fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } } }
I20260925 14:49:24.607415 11227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 248c97d99802445184efa4a8ccf309fc. Latest consensus state: current_term: 1 leader_uuid: "248c97d99802445184efa4a8ccf309fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "248c97d99802445184efa4a8ccf309fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45019 } } }
I20260925 14:49:24.607463 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11154
I20260925 14:49:24.607501 11226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:24.607513 11227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:24.607591 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:24.607703 11229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:24.607836 22327 external_mini_cluster.cc:1428] Setting key 58445dcef33410a5720f1c457d05a649
I20260925 14:49:24.608246 11229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:24.610080 11229 catalog_manager.cc:1347] Generated new cluster ID: 08f1d1e868ca408eb3b16689fee3a775
I20260925 14:49:24.610131 11229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:24.614359 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:24.608509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:55747 (local address 127.0.0.1:45019)
0925 14:49:24.608696 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:24.608706 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:24.608903 (+ 197us) server_negotiation.cc:407] Connection header received
0925 14:49:24.609208 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:24.609216 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:24.609281 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:24.609446 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:24.609673 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:24.610965 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:24.612094 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:24.612698 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:24.612788 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:24.613669 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:24.613673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:24.613673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:24.613873 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:24.613907 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:24.613920 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:24.613992 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:24.614141 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 14:49:24.614166 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260925 14:49:24.615340 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:24.620612 11229 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:24.620777 11241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:24.616877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:47943 (local address 127.0.0.1:45019)
0925 14:49:24.616907 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:24.616909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:24.617014 (+ 105us) server_negotiation.cc:407] Connection header received
0925 14:49:24.617090 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:24.617091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:24.617101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:24.617151 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:24.617409 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:24.618097 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:24.619065 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:24.619377 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:24.619435 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:24.620222 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:24.620224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:24.620224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:24.620229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:24.620244 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:24.620248 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:24.620286 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:24.620700 (+ 414us) server_negotiation.cc:299] Negotiation successful
0925 14:49:24.620715 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260925 14:49:24.621332 11229 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:24.627393 11229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 248c97d99802445184efa4a8ccf309fc: Generated new TSK 0
I20260925 14:49:24.627578 11229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:24.631669 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11154
W20260925 14:49:24.636297 11139 connection.cc:535] client connection to 127.0.0.1:45019 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45019): Connection reset by peer (error 104)
2026-09-25T14:49:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:24 dist-test-slave-d7lw krb5kdc[11253](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:24 dist-test-slave-d7lw krb5kdc[11253](info): set up 2 sockets
Sep 25 14:49:24 dist-test-slave-d7lw krb5kdc[11253](info): commencing operation
krb5kdc: starting...
W20260925 14:49:25.666731 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:25 dist-test-slave-d7lw krb5kdc[11253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347765, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:25.790509 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:25.872417 11266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:25.872648 11266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:25.872687 11266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:25.873870 11266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:25.873939 11266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:25.873965 11266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:25.873994 11266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:25.874018 11266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:25.875583 11266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:25.875877 11266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:25.876117 11266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:25.880956 11274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:25.880944 11271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:25.881057 11272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:25.881072 11266 server_base.cc:1034] running on GCE node
I20260925 14:49:25.881444 11266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:25.881794 11266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:25.882977 11266 hybrid_clock.cc:648] HybridClock initialized: now 1790347765882971 us; error 49 us; skew 500 ppm
Sep 25 14:49:25 dist-test-slave-d7lw krb5kdc[11253](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347765, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:25.888424 11266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:25.888859 11266 webserver.cc:460] Webserver started at http://127.0.0.1:44465/ using document root <none> and password file <none>
I20260925 14:49:25.889097 11266 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:25.889168 11266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:25.889324 11266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:25.890460 11266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "9a24f458719347db8a9a7400d087c8f7"
format_stamp: "Formatted at 2026-09-25 14:49:25 on dist-test-slave-d7lw"
server_key: "e1ff1980913a15107ff42896b459ee31"
server_key_iv: "10d23881c97cd2510b7ec19e18cd9040"
server_key_version: "encryptionkey@0"
I20260925 14:49:25.890821 11266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "9a24f458719347db8a9a7400d087c8f7"
format_stamp: "Formatted at 2026-09-25 14:49:25 on dist-test-slave-d7lw"
server_key: "e1ff1980913a15107ff42896b459ee31"
server_key_iv: "10d23881c97cd2510b7ec19e18cd9040"
server_key_version: "encryptionkey@0"
I20260925 14:49:25.892171 11266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:25.892920 11281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:25.893088 11266 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:25.893193 11266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "9a24f458719347db8a9a7400d087c8f7"
format_stamp: "Formatted at 2026-09-25 14:49:25 on dist-test-slave-d7lw"
server_key: "e1ff1980913a15107ff42896b459ee31"
server_key_iv: "10d23881c97cd2510b7ec19e18cd9040"
server_key_version: "encryptionkey@0"
I20260925 14:49:25.893266 11266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:25.896538 11266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:25.898415 11266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:25.898574 11266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:25.902423 11266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39373
I20260925 14:49:25.902482 11335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39373 every 8 connection(s)
I20260925 14:49:25.902815 11266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:25.903878 11336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:25.905115 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11266
I20260925 14:49:25.905256 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:25.905426 22327 external_mini_cluster.cc:1428] Setting key cbd533aabb103f3a55de02bc9e73c41b
I20260925 14:49:25.906451 11336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7: Bootstrap starting.
I20260925 14:49:25.907168 11336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:25.907495 11336 log.cc:824] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:25.907863 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:25.905943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:59383 (local address 127.0.0.1:39373)
0925 14:49:25.906121 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:25.906128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:25.906153 (+ 25us) server_negotiation.cc:407] Connection header received
0925 14:49:25.906578 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:25.906586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:25.906623 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:25.906733 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:25.907373 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:25.907376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:25.907377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:25.907381 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:25.907425 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:25.907440 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:25.907520 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:25.907576 (+ 56us) server_negotiation.cc:299] Negotiation successful
0925 14:49:25.907617 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260925 14:49:25.908319 11336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7: No bootstrap required, opened a new log
I20260925 14:49:25.908990 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:25.910444 11336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260925 14:49:25.910629 11336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:25.910667 11336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a24f458719347db8a9a7400d087c8f7, State: Initialized, Role: FOLLOWER
I20260925 14:49:25.910787 11336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260925 14:49:25.910887 11336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:25.910910 11336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:25.911019 11336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:25.911379 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:25.910370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:47731 (local address 127.0.0.1:39373)
0925 14:49:25.910411 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:25.910413 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:25.910469 (+ 56us) server_negotiation.cc:407] Connection header received
0925 14:49:25.910524 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:25.910525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:25.910529 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:25.910600 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:25.911189 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:25.911191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:25.911191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:25.911196 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:25.911209 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:25.911214 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:25.911280 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:25.911315 (+ 35us) server_negotiation.cc:299] Negotiation successful
0925 14:49:25.911327 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260925 14:49:25.911846 11336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260925 14:49:25.911949 11336 leader_election.cc:302] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a24f458719347db8a9a7400d087c8f7; no voters:
I20260925 14:49:25.912101 11336 leader_election.cc:288] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:25.912120 11346 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:25.912256 11346 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [term 1 LEADER]: Becoming Leader. State: Replica: 9a24f458719347db8a9a7400d087c8f7, State: Running, Role: LEADER
I20260925 14:49:25.912351 11336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:25.912406 11346 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260925 14:49:25.912815 11348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a24f458719347db8a9a7400d087c8f7. Latest consensus state: current_term: 1 leader_uuid: "9a24f458719347db8a9a7400d087c8f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } } }
I20260925 14:49:25.913069 11348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:25.912963 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a24f458719347db8a9a7400d087c8f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a24f458719347db8a9a7400d087c8f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } } }
I20260925 14:49:25.913441 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:25.913540 11355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:25.913995 11355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:25.915526 11355 catalog_manager.cc:1347] Generated new cluster ID: 661314e49ee24821a3b61d9b083d8961
I20260925 14:49:25.915575 11355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:25.943150 11355 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:25.943356 11355 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:25.957670 11355 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a24f458719347db8a9a7400d087c8f7: Generated new TSK 0
I20260925 14:49:25.957877 11355 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:25.965574 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11266
2026-09-25T14:49:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:26 dist-test-slave-d7lw krb5kdc[11367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:26 dist-test-slave-d7lw krb5kdc[11367](info): set up 2 sockets
Sep 25 14:49:26 dist-test-slave-d7lw krb5kdc[11367](info): commencing operation
krb5kdc: starting...
W20260925 14:49:27.003077 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:27.119939 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40265
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:27.198462 11380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:27.198691 11380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:27.198742 11380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:27.199851 11380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:27.199921 11380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:27.199944 11380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:27.199970 11380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:27.200001 11380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:27.201510 11380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40265
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:27.201774 11380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:27.202050 11380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:27.207002 11385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:27.206979 11386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:27.207007 11388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:27.207222 11380 server_base.cc:1034] running on GCE node
I20260925 14:49:27.207502 11380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:27.207834 11380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:27.209007 11380 hybrid_clock.cc:648] HybridClock initialized: now 1790347767208980 us; error 37 us; skew 500 ppm
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347767, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:27.211714 11380 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:27.212182 11380 webserver.cc:460] Webserver started at http://127.0.0.1:40561/ using document root <none> and password file <none>
I20260925 14:49:27.212445 11380 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:27.212512 11380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:27.212666 11380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:27.213712 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "8efd39c8ea4543e38d7571117332e9ea"
format_stamp: "Formatted at 2026-09-25 14:49:27 on dist-test-slave-d7lw"
server_key: "20d6a6a29feb7d2c84e41b389fd58b0e"
server_key_iv: "4f3987c25af85f36302fdd57b02ba475"
server_key_version: "encryptionkey@0"
I20260925 14:49:27.214084 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "8efd39c8ea4543e38d7571117332e9ea"
format_stamp: "Formatted at 2026-09-25 14:49:27 on dist-test-slave-d7lw"
server_key: "20d6a6a29feb7d2c84e41b389fd58b0e"
server_key_iv: "4f3987c25af85f36302fdd57b02ba475"
server_key_version: "encryptionkey@0"
I20260925 14:49:27.215377 11380 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260925 14:49:27.216138 11395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:27.216310 11380 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:27.216399 11380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "8efd39c8ea4543e38d7571117332e9ea"
format_stamp: "Formatted at 2026-09-25 14:49:27 on dist-test-slave-d7lw"
server_key: "20d6a6a29feb7d2c84e41b389fd58b0e"
server_key_iv: "4f3987c25af85f36302fdd57b02ba475"
server_key_version: "encryptionkey@0"
I20260925 14:49:27.216480 11380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:27.227520 11380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:27.229852 11380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:27.230051 11380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:27.234045 11380 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40265
I20260925 14:49:27.234086 11448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40265 every 8 connection(s)
I20260925 14:49:27.234441 11380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:27.235303 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:27.237578 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea: Bootstrap starting.
I20260925 14:49:27.238188 11449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:27.238463 11449 log.cc:824] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:27.239163 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea: No bootstrap required, opened a new log
I20260925 14:49:27.240832 11449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260925 14:49:27.240965 11449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:27.240986 11449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efd39c8ea4543e38d7571117332e9ea, State: Initialized, Role: FOLLOWER
I20260925 14:49:27.241182 11449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260925 14:49:27.241247 11449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:27.241286 11449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:27.241358 11449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:27.241894 11449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260925 14:49:27.241977 11449 leader_election.cc:302] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8efd39c8ea4543e38d7571117332e9ea; no voters:
I20260925 14:49:27.242183 11449 leader_election.cc:288] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:27.242257 11452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:27.242442 11452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [term 1 LEADER]: Becoming Leader. State: Replica: 8efd39c8ea4543e38d7571117332e9ea, State: Running, Role: LEADER
I20260925 14:49:27.242465 11449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:27.242544 11452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } }
I20260925 14:49:27.242913 11454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8efd39c8ea4543e38d7571117332e9ea. Latest consensus state: current_term: 1 leader_uuid: "8efd39c8ea4543e38d7571117332e9ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } } }
I20260925 14:49:27.243000 11454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:27.243064 11453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8efd39c8ea4543e38d7571117332e9ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efd39c8ea4543e38d7571117332e9ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40265 } } }
I20260925 14:49:27.243122 11453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:27.243253 11461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:27.243760 11461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:27.244226 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11380
I20260925 14:49:27.244345 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:27.244529 22327 external_mini_cluster.cc:1428] Setting key 0afc8c88b5c15706aece3112b5ffa124
I20260925 14:49:27.245584 11461 catalog_manager.cc:1347] Generated new cluster ID: f59fa59b060d4e6499c9b65e49e9eb4c
I20260925 14:49:27.245636 11461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347767, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 14:49:27.256100 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:27.245126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:50571 (local address 127.0.0.1:40265)
0925 14:49:27.245285 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:27.245294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:27.245328 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:49:27.245443 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:27.245450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:27.245508 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:27.245668 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:27.246406 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:27.247860 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:27.248900 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:27.249564 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:27.249679 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:27.252489 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:27.252494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:27.252494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:27.252506 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:27.254400 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:27.254858 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:27.254860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:27.254861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:27.254961 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:27.255193 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:27.255194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:27.255194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:27.255478 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:27.255644 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:27.255759 (+ 115us) server_negotiation.cc:299] Negotiation successful
0925 14:49:27.255879 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260925 14:49:27.257076 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:27.264549 11461 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:27.265079 11461 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:27.265654 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:27.258549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:60779 (local address 127.0.0.1:40265)
0925 14:49:27.258583 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:27.258584 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:27.258592 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:27.258645 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:27.258646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:27.258654 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:27.258713 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:27.259283 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:27.260092 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:27.261026 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:27.261343 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:27.261413 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:27.262723 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:27.262724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:27.262724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:27.262731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:27.264357 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:27.264802 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:27.264803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:27.264804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:27.264857 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:27.265088 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:27.265088 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:27.265089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:27.265207 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:27.265279 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:27.265503 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 14:49:27.265567 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 14:49:27.271032 11461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8efd39c8ea4543e38d7571117332e9ea: Generated new TSK 0
I20260925 14:49:27.271216 11461 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:27.277292 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11380
W20260925 14:49:27.282192 11365 connection.cc:535] client connection to 127.0.0.1:40265 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40265): Connection reset by peer (error 104)
2026-09-25T14:49:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11480](info): set up 2 sockets
Sep 25 14:49:27 dist-test-slave-d7lw krb5kdc[11480](info): commencing operation
krb5kdc: starting...
W20260925 14:49:28.311089 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:28 dist-test-slave-d7lw krb5kdc[11480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347768, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:28.428555 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:28.509212 11494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:28.509455 11494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:28.509493 11494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:28.510629 11494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:28.510696 11494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:28.510720 11494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:28.510743 11494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:28.510771 11494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:28.512249 11494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:28.512543 11494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:28.512768 11494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:28.517598 11500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:28.517606 11502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:28.517669 11494 server_base.cc:1034] running on GCE node
W20260925 14:49:28.517609 11499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:28.518054 11494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:28.518333 11494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:28.519500 11494 hybrid_clock.cc:648] HybridClock initialized: now 1790347768519480 us; error 39 us; skew 500 ppm
Sep 25 14:49:28 dist-test-slave-d7lw krb5kdc[11480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347768, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:28.521973 11494 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:28.522403 11494 webserver.cc:460] Webserver started at http://127.0.0.1:34321/ using document root <none> and password file <none>
I20260925 14:49:28.522647 11494 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:28.522693 11494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:28.522792 11494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:28.523931 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d"
format_stamp: "Formatted at 2026-09-25 14:49:28 on dist-test-slave-d7lw"
server_key: "4468d30ef59c9610560be1ff0455c191"
server_key_iv: "d831f9634e196a731acf6aade71c646e"
server_key_version: "encryptionkey@0"
I20260925 14:49:28.524237 11494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d"
format_stamp: "Formatted at 2026-09-25 14:49:28 on dist-test-slave-d7lw"
server_key: "4468d30ef59c9610560be1ff0455c191"
server_key_iv: "d831f9634e196a731acf6aade71c646e"
server_key_version: "encryptionkey@0"
I20260925 14:49:28.525504 11494 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:28.526207 11509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:28.526398 11494 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:28.526487 11494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d"
format_stamp: "Formatted at 2026-09-25 14:49:28 on dist-test-slave-d7lw"
server_key: "4468d30ef59c9610560be1ff0455c191"
server_key_iv: "d831f9634e196a731acf6aade71c646e"
server_key_version: "encryptionkey@0"
I20260925 14:49:28.526563 11494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:28.541344 11494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:28.543627 11494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:28.543737 11494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:28.547479 11494 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45443
I20260925 14:49:28.547533 11561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45443 every 8 connection(s)
I20260925 14:49:28.547900 11494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:28.548869 11562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:28.551177 11562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: Bootstrap starting.
I20260925 14:49:28.551796 11562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:28.552095 11562 log.cc:824] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:28.552803 11562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: No bootstrap required, opened a new log
I20260925 14:49:28.553474 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11494
I20260925 14:49:28.553598 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:28.553807 22327 external_mini_cluster.cc:1428] Setting key 6e42f924dfb6bc3a7c21cbd52e7febbb
I20260925 14:49:28.554903 11562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260925 14:49:28.555065 11562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:28.555130 11562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2517a3bfa8c2400c8ce9bc171ecdab1d, State: Initialized, Role: FOLLOWER
I20260925 14:49:28.555310 11562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260925 14:49:28.555400 11562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:28.555438 11562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:28.555491 11562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:28.556262 11562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260925 14:49:28.556370 11562 leader_election.cc:302] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2517a3bfa8c2400c8ce9bc171ecdab1d; no voters:
I20260925 14:49:28.556530 11562 leader_election.cc:288] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:28.556578 11567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:28.556723 11567 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [term 1 LEADER]: Becoming Leader. State: Replica: 2517a3bfa8c2400c8ce9bc171ecdab1d, State: Running, Role: LEADER
I20260925 14:49:28.556772 11562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:28.557418 11567 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } }
I20260925 14:49:28.558038 11568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2517a3bfa8c2400c8ce9bc171ecdab1d. Latest consensus state: current_term: 1 leader_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } } }
I20260925 14:49:28.558180 11569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2517a3bfa8c2400c8ce9bc171ecdab1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45443 } } }
I20260925 14:49:28.559648 11569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:28.559628 11568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d [sys.catalog]: This master's current role is: LEADER
W20260925 14:49:28.558563 11581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:28.559813 11581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:28.560012 11584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:28.560277 11584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:28.560563 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:28.554328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:55793 (local address 127.0.0.1:45443)
0925 14:49:28.554506 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:28.554516 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:28.554553 (+ 37us) server_negotiation.cc:407] Connection header received
0925 14:49:28.554887 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:28.554893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:28.554942 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:28.555084 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:28.555315 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:28.557038 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:28.558292 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:28.558836 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:28.558978 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:28.559716 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:28.559720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:28.559720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:28.559969 (+ 249us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:28.560013 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:28.560029 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:28.560111 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:28.560285 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 14:49:28.560316 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260925 14:49:28.561621 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:28.562055 11584 catalog_manager.cc:1347] Generated new cluster ID: 1105a3f83e614d99af2dee4477370d72
I20260925 14:49:28.562107 11584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:28.567642 11566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:28.563865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:33579 (local address 127.0.0.1:45443)
0925 14:49:28.563921 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:28.563923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:28.563932 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:28.563952 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:28.563953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:28.563964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:28.564023 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:28.564177 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:28.565216 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:28.566221 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:28.566555 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:28.566639 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:28.567358 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:28.567359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:28.567360 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:28.567365 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:28.567383 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:28.567388 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:28.567453 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:28.567523 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 14:49:28.567536 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 14:49:28.573323 11584 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:28.573853 11584 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:28.582548 11584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2517a3bfa8c2400c8ce9bc171ecdab1d: Generated new TSK 0
I20260925 14:49:28.582767 11584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:28.585098 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11494
W20260925 14:49:28.589615 11478 connection.cc:535] client connection to 127.0.0.1:45443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45443): Connection reset by peer (error 104)
2026-09-25T14:49:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1307 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:28 dist-test-slave-d7lw krb5kdc[11593](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:28 dist-test-slave-d7lw krb5kdc[11593](info): set up 2 sockets
Sep 25 14:49:28 dist-test-slave-d7lw krb5kdc[11593](info): commencing operation
krb5kdc: starting...
W20260925 14:49:29.618791 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:29 dist-test-slave-d7lw krb5kdc[11593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 14:49:29.732008 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:29.819212 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:29.819530 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:29.819617 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:29.821408 11606 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:29.821470 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:29.821496 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:29.821518 11606 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:29.821559 11606 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:29.823220 11606 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38065
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:29.823515 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:29.823746 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:29.828629 11612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:29.828641 11611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:29.828644 11614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:29.828675 11606 server_base.cc:1034] running on GCE node
I20260925 14:49:29.829129 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:29.829383 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:29.830555 11606 hybrid_clock.cc:648] HybridClock initialized: now 1790347769830537 us; error 29 us; skew 500 ppm
Sep 25 14:49:29 dist-test-slave-d7lw krb5kdc[11593](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:29.833127 11606 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 14:49:29.833621 11606 webserver.cc:460] Webserver started at http://127.0.0.1:43605/ using document root <none> and password file <none>
I20260925 14:49:29.833843 11606 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:29.833907 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:29.834074 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:29.835132 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "2b1263a6231d47438d6fa8aa4995a9ac"
format_stamp: "Formatted at 2026-09-25 14:49:29 on dist-test-slave-d7lw"
server_key: "7f652188b7a4d7741860b53aa4a23e1e"
server_key_iv: "dbc65de7356742465c322f739642e229"
server_key_version: "encryptionkey@0"
I20260925 14:49:29.835515 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "2b1263a6231d47438d6fa8aa4995a9ac"
format_stamp: "Formatted at 2026-09-25 14:49:29 on dist-test-slave-d7lw"
server_key: "7f652188b7a4d7741860b53aa4a23e1e"
server_key_iv: "dbc65de7356742465c322f739642e229"
server_key_version: "encryptionkey@0"
I20260925 14:49:29.836890 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:29.837605 11622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:29.837779 11606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:29.837877 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "2b1263a6231d47438d6fa8aa4995a9ac"
format_stamp: "Formatted at 2026-09-25 14:49:29 on dist-test-slave-d7lw"
server_key: "7f652188b7a4d7741860b53aa4a23e1e"
server_key_iv: "dbc65de7356742465c322f739642e229"
server_key_version: "encryptionkey@0"
I20260925 14:49:29.837963 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:29.841275 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:29.843493 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:29.843650 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:29.847611 11606 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38065
I20260925 14:49:29.847638 11674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38065 every 8 connection(s)
I20260925 14:49:29.847998 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:29.848902 11675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:29.851079 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac: Bootstrap starting.
I20260925 14:49:29.851670 11675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:29.851975 11675 log.cc:824] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:29.852653 11675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac: No bootstrap required, opened a new log
I20260925 14:49:29.854203 11675 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } }
I20260925 14:49:29.854348 11675 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:29.854372 11675 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b1263a6231d47438d6fa8aa4995a9ac, State: Initialized, Role: FOLLOWER
I20260925 14:49:29.854554 11675 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } }
I20260925 14:49:29.854614 11675 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:29.854660 11675 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:29.854758 11675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:29.855306 11675 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } }
I20260925 14:49:29.855413 11675 leader_election.cc:302] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b1263a6231d47438d6fa8aa4995a9ac; no voters:
I20260925 14:49:29.855571 11675 leader_election.cc:288] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:29.855597 11678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:29.855818 11678 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [term 1 LEADER]: Becoming Leader. State: Replica: 2b1263a6231d47438d6fa8aa4995a9ac, State: Running, Role: LEADER
I20260925 14:49:29.855871 11675 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:29.855907 11678 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } }
I20260925 14:49:29.856419 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11606
I20260925 14:49:29.856424 11679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } } }
I20260925 14:49:29.856529 11679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:29.856564 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:29.856798 22327 external_mini_cluster.cc:1428] Setting key 554f0ba29d8efd5e324a9f108e881434
I20260925 14:49:29.856792 11680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b1263a6231d47438d6fa8aa4995a9ac. Latest consensus state: current_term: 1 leader_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1263a6231d47438d6fa8aa4995a9ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38065 } } }
I20260925 14:49:29.856952 11680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:29.857093 11687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:29.857642 11687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:29.859283 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:29.857481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:53569 (local address 127.0.0.1:38065)
0925 14:49:29.857680 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:29.857708 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:29.857742 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:49:29.857858 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:29.857863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:29.857894 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:29.858032 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:29.858815 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:29.858819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:29.858819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:29.858824 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:29.858881 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:29.858895 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:29.858982 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:29.859062 (+ 80us) server_negotiation.cc:299] Negotiation successful
0925 14:49:29.859103 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260925 14:49:29.859579 11687 catalog_manager.cc:1347] Generated new cluster ID: 4dbf08b8aae540c9aa4b5a246d93908c
I20260925 14:49:29.859627 11687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:29.860143 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:29.865240 11697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:29.864063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.247:51211 (local address 127.0.0.1:38065)
0925 14:49:29.864100 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:29.864102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:29.864307 (+ 205us) server_negotiation.cc:407] Connection header received
0925 14:49:29.864366 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:29.864367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:29.864384 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:29.864431 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:29.865030 (+ 599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:29.865031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:29.865031 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:29.865035 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:29.865049 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:29.865053 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:29.865105 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:29.865159 (+ 54us) server_negotiation.cc:299] Negotiation successful
0925 14:49:29.865174 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:29.874881 11687 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:29.875064 11687 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:29.881325 11687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b1263a6231d47438d6fa8aa4995a9ac: Generated new TSK 0
I20260925 14:49:29.881539 11687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:29.885768 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11606
2026-09-25T14:49:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:29 dist-test-slave-d7lw krb5kdc[11706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:29 dist-test-slave-d7lw krb5kdc[11706](info): set up 2 sockets
Sep 25 14:49:29 dist-test-slave-d7lw krb5kdc[11706](info): commencing operation
krb5kdc: starting...
W20260925 14:49:30.919844 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:30 dist-test-slave-d7lw krb5kdc[11706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:31.036710 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38335
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:38737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:38335
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:31.118908 11719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:31.119143 11719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:31.119167 11719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:31.120456 11719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:31.120507 11719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:31.120522 11719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:31.120535 11719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:31.120548 11719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:31.122150 11719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:38737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:38335
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:38335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:31.122450 11719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:31.122710 11719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:31.127854 11727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:31.127815 11724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:31.127828 11725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:31.128157 11719 server_base.cc:1034] running on GCE node
I20260925 14:49:31.128486 11719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:31.128775 11719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:31.129961 11719 hybrid_clock.cc:648] HybridClock initialized: now 1790347771129951 us; error 33 us; skew 500 ppm
Sep 25 14:49:31 dist-test-slave-d7lw krb5kdc[11706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347771, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:31.135710 11719 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:31.136255 11719 webserver.cc:460] Webserver started at http://127.21.205.254:44925/ using document root <none> and password file <none>
I20260925 14:49:31.136479 11719 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:31.136529 11719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:31.136632 11719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:31.137768 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "c489966889994783891c994e64881c46"
format_stamp: "Formatted at 2026-09-25 14:49:31 on dist-test-slave-d7lw"
server_key: "dc1a4f6e435dadb9073adc2908cee506"
server_key_iv: "5214d8ed961a3d0ec6014af5900fe700"
server_key_version: "encryptionkey@0"
I20260925 14:49:31.138113 11719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "c489966889994783891c994e64881c46"
format_stamp: "Formatted at 2026-09-25 14:49:31 on dist-test-slave-d7lw"
server_key: "dc1a4f6e435dadb9073adc2908cee506"
server_key_iv: "5214d8ed961a3d0ec6014af5900fe700"
server_key_version: "encryptionkey@0"
I20260925 14:49:31.139536 11719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:31.140425 11734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:31.140584 11719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:31.140686 11719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "c489966889994783891c994e64881c46"
format_stamp: "Formatted at 2026-09-25 14:49:31 on dist-test-slave-d7lw"
server_key: "dc1a4f6e435dadb9073adc2908cee506"
server_key_iv: "5214d8ed961a3d0ec6014af5900fe700"
server_key_version: "encryptionkey@0"
I20260925 14:49:31.140758 11719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:31.150194 11719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:31.152915 11719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:31.153041 11719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:31.157131 11719 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:38335
I20260925 14:49:31.157174 11787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:38335 every 8 connection(s)
I20260925 14:49:31.157569 11719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:31.158545 11788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:31.161126 11788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46: Bootstrap starting.
I20260925 14:49:31.161906 11788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:31.161988 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11719
I20260925 14:49:31.162135 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:31.162277 11788 log.cc:824] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:31.162362 22327 external_mini_cluster.cc:1428] Setting key f6306544697787932d10f60322e4cf2c
I20260925 14:49:31.163268 11788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46: No bootstrap required, opened a new log
I20260925 14:49:31.166079 11788 raft_consensus.cc:348] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } }
I20260925 14:49:31.166275 11788 raft_consensus.cc:374] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:31.166309 11788 raft_consensus.cc:729] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c489966889994783891c994e64881c46, State: Initialized, Role: FOLLOWER
I20260925 14:49:31.166437 11788 consensus_queue.cc:260] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } }
I20260925 14:49:31.166508 11788 raft_consensus.cc:388] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:31.166536 11788 raft_consensus.cc:482] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:31.166580 11788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:31.167249 11788 raft_consensus.cc:504] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } }
I20260925 14:49:31.167397 11788 leader_election.cc:302] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c489966889994783891c994e64881c46; no voters:
I20260925 14:49:31.167593 11788 leader_election.cc:288] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:31.167665 11793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:31.167893 11793 raft_consensus.cc:686] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [term 1 LEADER]: Becoming Leader. State: Replica: c489966889994783891c994e64881c46, State: Running, Role: LEADER
I20260925 14:49:31.168059 11793 consensus_queue.cc:237] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } }
I20260925 14:49:31.167933 11788 sys_catalog.cc:564] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:31.169188 11794 sys_catalog.cc:455] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c489966889994783891c994e64881c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } } }
I20260925 14:49:31.169292 11794 sys_catalog.cc:458] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:31.169514 11806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:31.169636 11795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c489966889994783891c994e64881c46. Latest consensus state: current_term: 1 leader_uuid: "c489966889994783891c994e64881c46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c489966889994783891c994e64881c46" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 38335 } } }
I20260925 14:49:31.169705 11795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46 [sys.catalog]: This master's current role is: LEADER
Sep 25 14:49:31 dist-test-slave-d7lw krb5kdc[11706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:31.170179 11806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:31.172250 11806 catalog_manager.cc:1347] Generated new cluster ID: 5b6f93d4512a42f5a676f83137156985
I20260925 14:49:31.172323 11806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:31.174294 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:31.162947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58545 (local address 127.21.205.254:38335)
0925 14:49:31.163118 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:31.163135 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:31.163164 (+ 29us) server_negotiation.cc:407] Connection header received
0925 14:49:31.163669 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:31.163677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:31.163730 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:31.163933 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:31.164822 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:31.165743 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:31.166807 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:31.167220 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:31.167306 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:31.170531 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:31.170535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:31.170535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:31.170549 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:31.172680 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:31.173096 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:31.173100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:31.173100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:31.173202 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:31.173451 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:31.173452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:31.173452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:31.173709 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:31.173907 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:31.173997 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 14:49:31.174103 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260925 14:49:31.175457 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:31.186657 11792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:31.176918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45761 (local address 127.21.205.254:38335)
0925 14:49:31.176957 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:31.176958 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:31.180031 (+ 3073us) server_negotiation.cc:407] Connection header received
0925 14:49:31.180108 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:31.180109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:31.180120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:31.180170 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:31.180737 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:31.181283 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:31.182034 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:31.182281 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:31.182361 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:31.183674 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:31.183676 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:31.183676 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:31.183684 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:31.185260 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:31.185662 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:31.185664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:31.185664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:31.185727 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:31.185956 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:31.185957 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:31.185957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:31.186084 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:31.186216 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:31.186468 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 14:49:31.186536 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:31.193908 11806 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:31.194725 11806 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:31.208290 11806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c489966889994783891c994e64881c46: Generated new TSK 0
I20260925 14:49:31.208560 11806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:31.215039 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11719
W20260925 14:49:31.220009 11704 connection.cc:535] client connection to 127.21.205.254:38335 recv error: Network error: failed to read from TLS socket (remote: 127.21.205.254:38335): Connection reset by peer (error 104)
2026-09-25T14:49:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1329 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:31 dist-test-slave-d7lw krb5kdc[11819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:31 dist-test-slave-d7lw krb5kdc[11819](info): set up 2 sockets
Sep 25 14:49:31 dist-test-slave-d7lw krb5kdc[11819](info): commencing operation
krb5kdc: starting...
W20260925 14:49:32.271034 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:32 dist-test-slave-d7lw krb5kdc[11819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:32.394120 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33821
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:41507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:33821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:32.477604 11832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:32.477862 11832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:32.477890 11832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:32.479099 11832 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:32.479146 11832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:32.479161 11832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:32.479173 11832 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:32.479185 11832 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:32.480762 11832 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:41507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:33821
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:33821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:32.481000 11832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:32.481205 11832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:32.486370 11837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:32.486423 11832 server_base.cc:1034] running on GCE node
W20260925 14:49:32.486450 11838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:32.486388 11840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:32.486838 11832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:32.487178 11832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:32.488337 11832 hybrid_clock.cc:648] HybridClock initialized: now 1790347772488312 us; error 39 us; skew 500 ppm
Sep 25 14:49:32 dist-test-slave-d7lw krb5kdc[11819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:32.490957 11832 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:32.491400 11832 webserver.cc:460] Webserver started at http://127.21.205.254:34671/ using document root <none> and password file <none>
I20260925 14:49:32.491670 11832 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:32.491758 11832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:32.491926 11832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:32.493081 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "8b718ec3dd324a55909708b55f56aab9"
format_stamp: "Formatted at 2026-09-25 14:49:32 on dist-test-slave-d7lw"
server_key: "6e2327642bd3805425d1e90077026166"
server_key_iv: "1aa0f3306d1ce19f353f0ae77805a063"
server_key_version: "encryptionkey@0"
I20260925 14:49:32.493471 11832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "8b718ec3dd324a55909708b55f56aab9"
format_stamp: "Formatted at 2026-09-25 14:49:32 on dist-test-slave-d7lw"
server_key: "6e2327642bd3805425d1e90077026166"
server_key_iv: "1aa0f3306d1ce19f353f0ae77805a063"
server_key_version: "encryptionkey@0"
I20260925 14:49:32.494940 11832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:32.495756 11847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:32.496018 11832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:32.496119 11832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "8b718ec3dd324a55909708b55f56aab9"
format_stamp: "Formatted at 2026-09-25 14:49:32 on dist-test-slave-d7lw"
server_key: "6e2327642bd3805425d1e90077026166"
server_key_iv: "1aa0f3306d1ce19f353f0ae77805a063"
server_key_version: "encryptionkey@0"
I20260925 14:49:32.496183 11832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:32.505321 11832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:32.507978 11832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:32.508119 11832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:32.512297 11832 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:33821
I20260925 14:49:32.512336 11899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:33821 every 8 connection(s)
I20260925 14:49:32.512749 11832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:32.513762 11900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:32.516377 11900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9: Bootstrap starting.
I20260925 14:49:32.517024 11900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:32.517350 11900 log.cc:824] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:32.518152 11900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9: No bootstrap required, opened a new log
I20260925 14:49:32.519559 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11832
I20260925 14:49:32.519683 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:32.519918 22327 external_mini_cluster.cc:1428] Setting key 44090d4e01f9aa7e0ffbc32a5d284b4c
I20260925 14:49:32.520334 11900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } }
I20260925 14:49:32.520577 11900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:32.520666 11900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b718ec3dd324a55909708b55f56aab9, State: Initialized, Role: FOLLOWER
I20260925 14:49:32.520862 11900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } }
I20260925 14:49:32.520951 11900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:32.520984 11900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:32.521025 11900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:32.522320 11900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } }
I20260925 14:49:32.522437 11900 leader_election.cc:302] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b718ec3dd324a55909708b55f56aab9; no voters:
I20260925 14:49:32.522593 11900 leader_election.cc:288] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:32.522653 11905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:32.522840 11905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [term 1 LEADER]: Becoming Leader. State: Replica: 8b718ec3dd324a55909708b55f56aab9, State: Running, Role: LEADER
I20260925 14:49:32.522871 11900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:32.522992 11905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } }
I20260925 14:49:32.523437 11906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b718ec3dd324a55909708b55f56aab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } } }
I20260925 14:49:32.523536 11906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:32.523695 11907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b718ec3dd324a55909708b55f56aab9. Latest consensus state: current_term: 1 leader_uuid: "8b718ec3dd324a55909708b55f56aab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b718ec3dd324a55909708b55f56aab9" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 33821 } } }
I20260925 14:49:32.523751 11907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:32.524163 11914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:32.524761 11914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:32.526631 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:32.520521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60671 (local address 127.21.205.254:33821)
0925 14:49:32.520755 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:32.520765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:32.520799 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:49:32.520928 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:32.520936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:32.521001 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:32.521162 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:32.521397 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:32.522890 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:32.524185 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:32.524898 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:32.525041 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:32.525845 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:32.525849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:32.525849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:32.526092 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:32.526137 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:32.526155 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:32.526277 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:32.526366 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 14:49:32.526399 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":120,"threads_started":1}
I20260925 14:49:32.526661 11914 catalog_manager.cc:1347] Generated new cluster ID: 313edd1ad9c74f3186bfd6b235966369
I20260925 14:49:32.526706 11914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:32.527562 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:32.534739 11904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:32.528985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51587 (local address 127.21.205.254:33821)
0925 14:49:32.529026 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:32.529028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:32.531746 (+ 2718us) server_negotiation.cc:407] Connection header received
0925 14:49:32.531860 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:32.531862 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:32.531874 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:32.531932 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:32.532088 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:32.532596 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:32.533351 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:32.533590 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:32.533706 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:32.534449 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:32.534451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:32.534451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:32.534456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:32.534471 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:32.534476 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:32.534534 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:32.534609 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 14:49:32.534623 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:32.550058 11914 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:32.550671 11914 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:32.569214 11914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b718ec3dd324a55909708b55f56aab9: Generated new TSK 0
I20260925 14:49:32.569470 11914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:32.578364 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11832
W20260925 14:49:32.583467 11817 connection.cc:535] client connection to 127.21.205.254:33821 recv error: Network error: failed to read from TLS socket (remote: 127.21.205.254:33821): Connection reset by peer (error 104)
2026-09-25T14:49:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1363 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:32 dist-test-slave-d7lw krb5kdc[11932](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:32 dist-test-slave-d7lw krb5kdc[11932](info): set up 2 sockets
Sep 25 14:49:32 dist-test-slave-d7lw krb5kdc[11932](info): commencing operation
krb5kdc: starting...
W20260925 14:49:33.614650 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:33 dist-test-slave-d7lw krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:33.726883 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43473
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:43473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:33.813403 11945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:33.813640 11945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:33.813676 11945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:33.814821 11945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:33.814884 11945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:33.814909 11945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:33.814936 11945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:33.814958 11945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:33.816474 11945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:43473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:43473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:33.816850 11945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:33.817117 11945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:49:33.822065 11945 server_base.cc:1034] running on GCE node
W20260925 14:49:33.822031 11950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:33.822046 11953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:33.822023 11951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:33.822597 11945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:33.822902 11945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:33.824064 11945 hybrid_clock.cc:648] HybridClock initialized: now 1790347773824052 us; error 57 us; skew 500 ppm
Sep 25 14:49:33 dist-test-slave-d7lw krb5kdc[11932](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347773, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:33.826486 11945 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:33.826951 11945 webserver.cc:460] Webserver started at http://127.21.205.254:44945/ using document root <none> and password file <none>
I20260925 14:49:33.827174 11945 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:33.827239 11945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:33.827385 11945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:33.828470 11945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "a964c4fd955641a5b717b96ff3670e8e"
format_stamp: "Formatted at 2026-09-25 14:49:33 on dist-test-slave-d7lw"
server_key: "482f4f966a56c5a6c2e63aa981af8b52"
server_key_iv: "b308862a54189e62fd5e4705eb894d08"
server_key_version: "encryptionkey@0"
I20260925 14:49:33.828825 11945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "a964c4fd955641a5b717b96ff3670e8e"
format_stamp: "Formatted at 2026-09-25 14:49:33 on dist-test-slave-d7lw"
server_key: "482f4f966a56c5a6c2e63aa981af8b52"
server_key_iv: "b308862a54189e62fd5e4705eb894d08"
server_key_version: "encryptionkey@0"
I20260925 14:49:33.830197 11945 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:33.830916 11960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:33.831096 11945 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:33.831176 11945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "a964c4fd955641a5b717b96ff3670e8e"
format_stamp: "Formatted at 2026-09-25 14:49:33 on dist-test-slave-d7lw"
server_key: "482f4f966a56c5a6c2e63aa981af8b52"
server_key_iv: "b308862a54189e62fd5e4705eb894d08"
server_key_version: "encryptionkey@0"
I20260925 14:49:33.831230 11945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:33.834527 11945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:33.837297 11945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:33.837438 11945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:33.841733 11945 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:43473
I20260925 14:49:33.841773 12012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:43473 every 8 connection(s)
I20260925 14:49:33.842121 11945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:33.843155 12013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:33.845467 12013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: Bootstrap starting.
I20260925 14:49:33.846036 12013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:33.846275 12013 log.cc:824] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:33.846917 12013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: No bootstrap required, opened a new log
I20260925 14:49:33.848636 12013 raft_consensus.cc:348] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } }
I20260925 14:49:33.848788 12013 raft_consensus.cc:374] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:33.848811 12013 raft_consensus.cc:729] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a964c4fd955641a5b717b96ff3670e8e, State: Initialized, Role: FOLLOWER
I20260925 14:49:33.848920 12013 consensus_queue.cc:260] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } }
I20260925 14:49:33.848973 12013 raft_consensus.cc:388] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:33.848995 12013 raft_consensus.cc:482] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:33.849038 12013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:33.849807 12013 raft_consensus.cc:504] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } }
I20260925 14:49:33.849895 12013 leader_election.cc:302] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a964c4fd955641a5b717b96ff3670e8e; no voters:
I20260925 14:49:33.850013 12013 leader_election.cc:288] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:33.850039 12016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:33.850207 12016 raft_consensus.cc:686] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [term 1 LEADER]: Becoming Leader. State: Replica: a964c4fd955641a5b717b96ff3670e8e, State: Running, Role: LEADER
I20260925 14:49:33.850297 12013 sys_catalog.cc:564] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:33.850373 12016 consensus_queue.cc:237] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } }
I20260925 14:49:33.850767 12017 sys_catalog.cc:455] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a964c4fd955641a5b717b96ff3670e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } } }
I20260925 14:49:33.850871 12017 sys_catalog.cc:458] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:33.851191 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 11945
I20260925 14:49:33.851322 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:33.851562 22327 external_mini_cluster.cc:1428] Setting key 620565bc407cef8ce8cc1083ab85a178
I20260925 14:49:33.850972 12018 sys_catalog.cc:455] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a964c4fd955641a5b717b96ff3670e8e. Latest consensus state: current_term: 1 leader_uuid: "a964c4fd955641a5b717b96ff3670e8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a964c4fd955641a5b717b96ff3670e8e" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 43473 } } }
I20260925 14:49:33.852559 12018 sys_catalog.cc:458] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e [sys.catalog]: This master's current role is: LEADER
W20260925 14:49:33.852169 12031 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:33.852660 12031 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:33.852730 12032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:33.852967 12032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:33.853981 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:33.852234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51903 (local address 127.21.205.254:43473)
0925 14:49:33.852465 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:33.852473 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:33.852507 (+ 34us) server_negotiation.cc:407] Connection header received
0925 14:49:33.852619 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:33.852625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:33.852655 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:33.852809 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:33.853523 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:33.853526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:33.853526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:33.853530 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:33.853565 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:33.853578 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:33.853647 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:33.853738 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 14:49:33.853783 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":119,"threads_started":1}
I20260925 14:49:33.854620 12032 catalog_manager.cc:1347] Generated new cluster ID: 4998bdf814d949078d9b2bc71316623a
I20260925 14:49:33.854715 12032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:33.854882 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:33.861449 12035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:33.860104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42321 (local address 127.21.205.254:43473)
0925 14:49:33.860145 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:33.860146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:33.860360 (+ 214us) server_negotiation.cc:407] Connection header received
0925 14:49:33.860442 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:33.860443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:33.860449 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:33.860511 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:33.861164 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:33.861165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:33.861165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:33.861170 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:33.861186 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:33.861191 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:33.861268 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:33.861332 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 14:49:33.861349 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:49:33.863917 12032 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:33.864077 12032 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:33.866714 12032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a964c4fd955641a5b717b96ff3670e8e: Generated new TSK 0
I20260925 14:49:33.866892 12032 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:33.868809 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 11945
2026-09-25T14:49:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1289 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:33 dist-test-slave-d7lw krb5kdc[12044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:33 dist-test-slave-d7lw krb5kdc[12044](info): set up 2 sockets
Sep 25 14:49:33 dist-test-slave-d7lw krb5kdc[12044](info): commencing operation
krb5kdc: starting...
W20260925 14:49:34.906629 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:34 dist-test-slave-d7lw krb5kdc[12044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:35.020522 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:44881
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:46829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:44881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:35.100571 12057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:35.100839 12057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:35.100879 12057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:35.102054 12057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:35.102124 12057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:35.102149 12057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:35.102170 12057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:35.102200 12057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:35.103690 12057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:46829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:44881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:44881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:35.103995 12057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:35.104239 12057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:35.109180 12062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:35.109210 12065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:35.109409 12057 server_base.cc:1034] running on GCE node
W20260925 14:49:35.109479 12063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:35.109725 12057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:35.110041 12057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:35.111199 12057 hybrid_clock.cc:648] HybridClock initialized: now 1790347775111185 us; error 29 us; skew 500 ppm
Sep 25 14:49:35 dist-test-slave-d7lw krb5kdc[12044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347775, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:35.113718 12057 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:35.114096 12057 webserver.cc:460] Webserver started at http://127.21.205.254:40165/ using document root <none> and password file <none>
I20260925 14:49:35.114295 12057 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:35.114336 12057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:35.114419 12057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:35.115468 12057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "c0f85254e56f4c4580ed5daf20ae8a23"
format_stamp: "Formatted at 2026-09-25 14:49:35 on dist-test-slave-d7lw"
server_key: "81d30054341b93d371f6585543966489"
server_key_iv: "66ddb3282673fa976c49f7470865f4ba"
server_key_version: "encryptionkey@0"
I20260925 14:49:35.115772 12057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "c0f85254e56f4c4580ed5daf20ae8a23"
format_stamp: "Formatted at 2026-09-25 14:49:35 on dist-test-slave-d7lw"
server_key: "81d30054341b93d371f6585543966489"
server_key_iv: "66ddb3282673fa976c49f7470865f4ba"
server_key_version: "encryptionkey@0"
I20260925 14:49:35.117110 12057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:35.117867 12072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:35.118052 12057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260925 14:49:35.118170 12057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "c0f85254e56f4c4580ed5daf20ae8a23"
format_stamp: "Formatted at 2026-09-25 14:49:35 on dist-test-slave-d7lw"
server_key: "81d30054341b93d371f6585543966489"
server_key_iv: "66ddb3282673fa976c49f7470865f4ba"
server_key_version: "encryptionkey@0"
I20260925 14:49:35.118247 12057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:35.133611 12057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:35.135479 12057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:35.135641 12057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:35.139575 12057 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:44881
I20260925 14:49:35.139638 12124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:44881 every 8 connection(s)
I20260925 14:49:35.140033 12057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:35.140923 12125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:35.143119 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23: Bootstrap starting.
I20260925 14:49:35.143689 12125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:35.144016 12125 log.cc:824] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:35.144708 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23: No bootstrap required, opened a new log
I20260925 14:49:35.145119 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12057
I20260925 14:49:35.145241 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:35.145419 22327 external_mini_cluster.cc:1428] Setting key abf92a7e1e31b9f95bdc727f69bc4ea3
I20260925 14:49:35.146770 12125 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } }
I20260925 14:49:35.146903 12125 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:35.146924 12125 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f85254e56f4c4580ed5daf20ae8a23, State: Initialized, Role: FOLLOWER
I20260925 14:49:35.147081 12125 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } }
I20260925 14:49:35.147173 12125 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:35.147207 12125 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:35.147258 12125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:35.148262 12125 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } }
I20260925 14:49:35.148347 12125 leader_election.cc:302] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0f85254e56f4c4580ed5daf20ae8a23; no voters:
I20260925 14:49:35.148545 12125 leader_election.cc:288] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:35.148591 12130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:35.148744 12130 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [term 1 LEADER]: Becoming Leader. State: Replica: c0f85254e56f4c4580ed5daf20ae8a23, State: Running, Role: LEADER
I20260925 14:49:35.148824 12125 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:35.148882 12130 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } }
I20260925 14:49:35.149271 12130 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0f85254e56f4c4580ed5daf20ae8a23. Latest consensus state: current_term: 1 leader_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } } }
I20260925 14:49:35.149281 12132 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0f85254e56f4c4580ed5daf20ae8a23" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 44881 } } }
I20260925 14:49:35.149420 12130 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:35.149428 12132 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:35.149859 12139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:35.150403 12139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:35.151959 12139 catalog_manager.cc:1347] Generated new cluster ID: 1ab4a463634e4b74a19e772233730785
I20260925 14:49:35.152014 12139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 25 14:49:35 dist-test-slave-d7lw krb5kdc[12044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:35.156611 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:35.145933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41097 (local address 127.21.205.254:44881)
0925 14:49:35.146118 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:35.146128 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:35.146159 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:49:35.146772 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:35.146780 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:35.146833 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:35.147015 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:35.147875 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:35.149089 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:35.150253 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:35.150826 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:35.150933 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:35.153452 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:35.153456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:35.153456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:35.153467 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:35.155153 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:35.155465 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:35.155468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:35.155468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:35.155566 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:35.155738 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:35.155739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:35.155739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:35.156043 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:35.156198 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:35.156285 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 14:49:35.156418 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260925 14:49:35.157627 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:35.165697 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:35.159081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42845 (local address 127.21.205.254:44881)
0925 14:49:35.159122 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:35.159124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:35.159216 (+ 92us) server_negotiation.cc:407] Connection header received
0925 14:49:35.159275 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:35.159276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:35.159284 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:35.159329 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:35.159947 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:35.160408 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:35.161382 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:35.161579 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:35.161651 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:35.162946 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:35.162947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:35.162948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:35.162966 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:35.164552 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:35.164933 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:35.164934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:35.164934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:35.164985 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:35.165188 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:35.165189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:35.165189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:35.165300 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:35.165384 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:35.165529 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 14:49:35.165600 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:49:35.180694 12139 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:35.181440 12139 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:35.190773 12139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0f85254e56f4c4580ed5daf20ae8a23: Generated new TSK 0
I20260925 14:49:35.190976 12139 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:35.196655 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12057
2026-09-25T14:49:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:35 dist-test-slave-d7lw krb5kdc[12156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:35 dist-test-slave-d7lw krb5kdc[12156](info): set up 2 sockets
Sep 25 14:49:35 dist-test-slave-d7lw krb5kdc[12156](info): commencing operation
krb5kdc: starting...
W20260925 14:49:36.264043 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:36 dist-test-slave-d7lw krb5kdc[12156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:36.378502 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34679
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:34717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:34679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:36.459347 12169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:36.459556 12169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:36.459579 12169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:36.460707 12169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:36.460745 12169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:36.460759 12169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:36.460772 12169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:36.460784 12169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:36.462183 12169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:34717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:34679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:34679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:36.462387 12169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:36.462554 12169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:36.467464 12174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:36.467473 12176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:36.467486 12178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:36.467648 12169 server_base.cc:1034] running on GCE node
I20260925 14:49:36.467984 12169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:36.468335 12169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:36.469506 12169 hybrid_clock.cc:648] HybridClock initialized: now 1790347776469491 us; error 45 us; skew 500 ppm
Sep 25 14:49:36 dist-test-slave-d7lw krb5kdc[12156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347776, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:36.478948 12169 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:36.479447 12169 webserver.cc:460] Webserver started at http://127.21.205.254:43157/ using document root <none> and password file <none>
I20260925 14:49:36.479712 12169 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:36.479785 12169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:36.479966 12169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:36.481020 12169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "bc98984617aa40a1b3b9d9fcc78d8845"
format_stamp: "Formatted at 2026-09-25 14:49:36 on dist-test-slave-d7lw"
server_key: "9134a8c987b197f71e0abeded6789204"
server_key_iv: "7d76324be68c9fc3d5849886ec272240"
server_key_version: "encryptionkey@0"
I20260925 14:49:36.481393 12169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "bc98984617aa40a1b3b9d9fcc78d8845"
format_stamp: "Formatted at 2026-09-25 14:49:36 on dist-test-slave-d7lw"
server_key: "9134a8c987b197f71e0abeded6789204"
server_key_iv: "7d76324be68c9fc3d5849886ec272240"
server_key_version: "encryptionkey@0"
I20260925 14:49:36.482713 12169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:36.483438 12185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:36.483624 12169 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:36.483726 12169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "bc98984617aa40a1b3b9d9fcc78d8845"
format_stamp: "Formatted at 2026-09-25 14:49:36 on dist-test-slave-d7lw"
server_key: "9134a8c987b197f71e0abeded6789204"
server_key_iv: "7d76324be68c9fc3d5849886ec272240"
server_key_version: "encryptionkey@0"
I20260925 14:49:36.483834 12169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:36.494652 12169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:36.496896 12169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:36.497054 12169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:36.501168 12169 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:34679
I20260925 14:49:36.501196 12238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:34679 every 8 connection(s)
I20260925 14:49:36.501557 12169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:36.502399 12239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:36.503325 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12169
I20260925 14:49:36.503446 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:36.503631 22327 external_mini_cluster.cc:1428] Setting key bb1e82e3ad9bbddd342094f4fc52b82e
I20260925 14:49:36.505309 12239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: Bootstrap starting.
I20260925 14:49:36.505998 12239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:36.506317 12239 log.cc:824] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:36.507026 12239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: No bootstrap required, opened a new log
I20260925 14:49:36.509302 12239 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } }
I20260925 14:49:36.509503 12239 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:36.509559 12239 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98984617aa40a1b3b9d9fcc78d8845, State: Initialized, Role: FOLLOWER
I20260925 14:49:36.509748 12239 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } }
I20260925 14:49:36.509835 12239 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:36.509902 12239 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:36.509963 12239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:36.510044 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:36.504163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55861 (local address 127.21.205.254:34679)
0925 14:49:36.504354 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:36.504363 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:36.504396 (+ 33us) server_negotiation.cc:407] Connection header received
0925 14:49:36.505026 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:36.505034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:36.505102 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:36.505252 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:36.505459 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:36.506823 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:36.507640 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:36.508284 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:36.508416 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:36.509037 (+ 621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:36.509042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:36.509042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:36.509239 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:36.509275 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:36.509290 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:36.509358 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:36.509457 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 14:49:36.509784 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":79,"threads_started":1}
I20260925 14:49:36.510702 12239 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } }
I20260925 14:49:36.510821 12239 leader_election.cc:302] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc98984617aa40a1b3b9d9fcc78d8845; no voters:
I20260925 14:49:36.510985 12239 leader_election.cc:288] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:36.511047 12244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:36.511150 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:36.511243 12244 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [term 1 LEADER]: Becoming Leader. State: Replica: bc98984617aa40a1b3b9d9fcc78d8845, State: Running, Role: LEADER
I20260925 14:49:36.511263 12239 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:36.511962 12244 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } }
I20260925 14:49:36.512547 12246 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } } }
I20260925 14:49:36.512639 12246 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:36.512733 12245 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc98984617aa40a1b3b9d9fcc78d8845. Latest consensus state: current_term: 1 leader_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc98984617aa40a1b3b9d9fcc78d8845" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 34679 } } }
I20260925 14:49:36.513007 12245 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845 [sys.catalog]: This master's current role is: LEADER
W20260925 14:49:36.513197 12262 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:36.513247 12262 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:36.513514 12264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:36.513749 12264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:36.515285 12264 catalog_manager.cc:1347] Generated new cluster ID: 081e29a40bf343369a37eca09fdcf24c
I20260925 14:49:36.515337 12264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:36.516583 12242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:36.512648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46507 (local address 127.21.205.254:34679)
0925 14:49:36.512681 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:36.512682 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:36.513347 (+ 665us) server_negotiation.cc:407] Connection header received
0925 14:49:36.513418 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:36.513419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:36.513430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:36.513479 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:36.513672 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:36.514386 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:36.515317 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:36.515616 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:36.515654 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:36.516357 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:36.516358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:36.516358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:36.516363 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:36.516374 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:36.516378 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:36.516403 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:36.516495 (+ 92us) server_negotiation.cc:299] Negotiation successful
0925 14:49:36.516516 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260925 14:49:36.520605 12264 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:36.521319 12264 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:36.527231 12264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc98984617aa40a1b3b9d9fcc78d8845: Generated new TSK 0
I20260925 14:49:36.527391 12264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:36.533228 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12169
2026-09-25T14:49:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1336 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:36 dist-test-slave-d7lw krb5kdc[12270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:36 dist-test-slave-d7lw krb5kdc[12270](info): set up 2 sockets
Sep 25 14:49:36 dist-test-slave-d7lw krb5kdc[12270](info): commencing operation
krb5kdc: starting...
W20260925 14:49:37.558827 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:37 dist-test-slave-d7lw krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:37.672055 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:36459
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:36459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:37.755731 12284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:37.756017 12284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:37.756054 12284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:37.757187 12284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:37.757249 12284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:37.757275 12284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:37.757305 12284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:37.757334 12284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:37.758772 12284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:36459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:36459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:37.759032 12284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:37.759262 12284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:37.764031 12292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:37.764042 12289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:37.764068 12290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:37.764355 12284 server_base.cc:1034] running on GCE node
I20260925 14:49:37.764585 12284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:37.764833 12284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:37.766011 12284 hybrid_clock.cc:648] HybridClock initialized: now 1790347777765997 us; error 25 us; skew 500 ppm
Sep 25 14:49:37 dist-test-slave-d7lw krb5kdc[12270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347777, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:37.768533 12284 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:37.769001 12284 webserver.cc:460] Webserver started at http://127.21.205.254:35911/ using document root <none> and password file <none>
I20260925 14:49:37.769248 12284 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:37.769317 12284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:37.769465 12284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:37.770475 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "d5d7abef57234b10b60c8971455b658b"
format_stamp: "Formatted at 2026-09-25 14:49:37 on dist-test-slave-d7lw"
server_key: "d15f86d28d56316387517cbd41486b70"
server_key_iv: "66ccf7e1163477b5c2a4536219dab700"
server_key_version: "encryptionkey@0"
I20260925 14:49:37.770823 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "d5d7abef57234b10b60c8971455b658b"
format_stamp: "Formatted at 2026-09-25 14:49:37 on dist-test-slave-d7lw"
server_key: "d15f86d28d56316387517cbd41486b70"
server_key_iv: "66ccf7e1163477b5c2a4536219dab700"
server_key_version: "encryptionkey@0"
I20260925 14:49:37.772151 12284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260925 14:49:37.772855 12299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:37.773026 12284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260925 14:49:37.773130 12284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "d5d7abef57234b10b60c8971455b658b"
format_stamp: "Formatted at 2026-09-25 14:49:37 on dist-test-slave-d7lw"
server_key: "d15f86d28d56316387517cbd41486b70"
server_key_iv: "66ccf7e1163477b5c2a4536219dab700"
server_key_version: "encryptionkey@0"
I20260925 14:49:37.773205 12284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:37.776935 12284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:37.779217 12284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:37.779381 12284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:37.783303 12284 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:36459
I20260925 14:49:37.783331 12352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:36459 every 8 connection(s)
I20260925 14:49:37.783643 12284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:37.784634 12353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:37.786274 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12284
I20260925 14:49:37.786404 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:37.786597 22327 external_mini_cluster.cc:1428] Setting key fb75acf8a77c1b49ad7b56976b62415a
I20260925 14:49:37.787173 12353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b: Bootstrap starting.
I20260925 14:49:37.787827 12353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:37.788128 12353 log.cc:824] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:37.788938 12353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b: No bootstrap required, opened a new log
I20260925 14:49:37.789098 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:37.787120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46189 (local address 127.21.205.254:36459)
0925 14:49:37.787289 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:37.787298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:37.787334 (+ 36us) server_negotiation.cc:407] Connection header received
0925 14:49:37.787753 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:37.787759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:37.787836 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:37.787978 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:37.788656 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:37.788659 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:37.788660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:37.788664 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:37.788709 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:37.788723 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:37.788823 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:37.788856 (+ 33us) server_negotiation.cc:299] Negotiation successful
0925 14:49:37.788895 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
I20260925 14:49:37.790019 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:37.791522 12353 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } }
I20260925 14:49:37.791697 12353 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 14:49:37.791739 12353 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d7abef57234b10b60c8971455b658b, State: Initialized, Role: FOLLOWER
I20260925 14:49:37.791952 12353 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } }
I20260925 14:49:37.792068 12353 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 14:49:37.792106 12353 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 14:49:37.792155 12353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:37.792666 12356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:37.791612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59649 (local address 127.21.205.254:36459)
0925 14:49:37.791650 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:37.791652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:37.791751 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:49:37.791859 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:37.791860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:37.791865 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:37.791923 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:37.792462 (+ 539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:37.792463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:37.792463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 14:49:37.792466 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:37.792477 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 14:49:37.792481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:37.792524 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:37.792577 (+ 53us) server_negotiation.cc:299] Negotiation successful
0925 14:49:37.792587 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:49:37.792874 12353 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } }
I20260925 14:49:37.792976 12353 leader_election.cc:302] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5d7abef57234b10b60c8971455b658b; no voters:
I20260925 14:49:37.793177 12353 leader_election.cc:288] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 14:49:37.793206 12363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:37.793321 12363 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [term 1 LEADER]: Becoming Leader. State: Replica: d5d7abef57234b10b60c8971455b658b, State: Running, Role: LEADER
I20260925 14:49:37.793433 12353 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:37.793450 12363 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } }
I20260925 14:49:37.793833 12364 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5d7abef57234b10b60c8971455b658b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } } }
I20260925 14:49:37.793851 12365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d7abef57234b10b60c8971455b658b. Latest consensus state: current_term: 1 leader_uuid: "d5d7abef57234b10b60c8971455b658b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d7abef57234b10b60c8971455b658b" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 36459 } } }
I20260925 14:49:37.793992 12365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:37.794188 12364 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:37.794242 12372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:37.794953 12372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:37.796464 12372 catalog_manager.cc:1347] Generated new cluster ID: 6267d970c88c4deda615837422a91432
I20260925 14:49:37.796517 12372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:37.801137 12372 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:37.801333 12372 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:37.808514 12372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5d7abef57234b10b60c8971455b658b: Generated new TSK 0
I20260925 14:49:37.808717 12372 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 14:49:37.812845 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12284
2026-09-25T14:49:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1279 ms)
[----------] 18 tests from AuthTokenIssuingTest (23755 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:37 dist-test-slave-d7lw krb5kdc[12384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:37 dist-test-slave-d7lw krb5kdc[12384](info): set up 2 sockets
Sep 25 14:49:37 dist-test-slave-d7lw krb5kdc[12384](info): commencing operation
krb5kdc: starting...
W20260925 14:49:38.854969 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.001s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:38 dist-test-slave-d7lw krb5kdc[12384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:38.967638 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42957
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:39.046378 12397 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:39.046584 12397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:39.046609 12397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:39.046629 12397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:39.047729 12397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:39.047765 12397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:39.047801 12397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:39.047816 12397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:39.047827 12397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:39.049253 12397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:42957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:39.049450 12397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:39.049619 12397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:39.054450 12403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:39.054486 12402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:39.054491 12405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:39.054800 12397 server_base.cc:1034] running on GCE node
I20260925 14:49:39.055094 12397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:39.055428 12397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:39.056586 12397 hybrid_clock.cc:648] HybridClock initialized: now 1790347779056565 us; error 39 us; skew 500 ppm
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:39.059010 12397 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:39.059489 12397 webserver.cc:460] Webserver started at http://127.21.205.254:44947/ using document root <none> and password file <none>
I20260925 14:49:39.059719 12397 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:39.059809 12397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:39.059965 12397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:39.061060 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "14216eed6c7a0cfd0525a3727e025a7e"
server_key_iv: "73f834be0dc70264c925f3167857f72d"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.061437 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "14216eed6c7a0cfd0525a3727e025a7e"
server_key_iv: "73f834be0dc70264c925f3167857f72d"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.062788 12397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:39.063499 12412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:39.063670 12397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:39.063771 12397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "14216eed6c7a0cfd0525a3727e025a7e"
server_key_iv: "73f834be0dc70264c925f3167857f72d"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.063880 12397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:39.086373 12397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:39.088428 12397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:39.088608 12397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:39.092866 12397 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:42957
I20260925 14:49:39.092901 12464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:42957 every 8 connection(s)
I20260925 14:49:39.093283 12397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:39.094225 12465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:39.095866 12465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:39.102360 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12397
I20260925 14:49:39.102497 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:39.102700 22327 external_mini_cluster.cc:1428] Setting key 3e0b44c7465026d72f0f895854287054
I20260925 14:49:39.106633 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.096655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.21.205.254:42957)
0925 14:49:39.096824 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.096835 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.096971 (+ 136us) server_negotiation.cc:407] Connection header received
0925 14:49:39.097604 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.097606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.097633 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.097735 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.098328 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.099163 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.100129 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.100449 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.100590 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.102944 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.102954 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.102954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.102963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.104814 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.105367 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.105370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.105371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.105465 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.105755 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.105756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.105756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.106007 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.106081 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.106324 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.106429 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260925 14:49:39.106745 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.096558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42957 (local address 127.0.0.1:49710)
0925 14:49:39.096813 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.096861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.097401 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.097741 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.097744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.098053 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.098215 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.098219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.099267 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.099268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.100024 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.100026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.100554 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.100555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.100629 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.101239 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.101248 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.102852 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.104907 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.104916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.104919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.105228 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.105544 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.105545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.105545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.105665 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.106086 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.106093 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.106178 (+ 85us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.106292 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.106544 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":63,"thread_start_us":112,"threads_started":1}
I20260925 14:49:39.107568 12465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:39.108028 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.107873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54238)
0925 14:49:39.107931 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.107990 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 14:49:39.108194 12416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
I20260925 14:49:39.108665 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.108545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54244)
0925 14:49:39.108599 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.108633 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:49:39.108851 12465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:35445: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.205.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
I20260925 14:49:39.128212 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:35445
--webserver_interface=127.21.205.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:49:39.158146 12465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } attempt: 1
I20260925 14:49:39.158778 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.158616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54256)
0925 14:49:39.158684 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.158736 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 14:49:39.159200 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.159098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54272)
0925 14:49:39.159147 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.159179 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 14:49:39.159317 12465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:35445: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
W20260925 14:49:39.212348 12472 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:39.212604 12472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:39.212622 12472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:39.212682 12472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:39.214481 12472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:39.214599 12472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:39.214636 12472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:39.214668 12472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:39.214699 12472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:39.217046 12472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:35445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.205.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:39.217494 12472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:39.217804 12472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:39.225405 12480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:39.225399 12477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:39.225406 12478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:39.225900 12472 server_base.cc:1034] running on GCE node
I20260925 14:49:39.226186 12472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:39.226548 12472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:39.227453 12465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } attempt: 2
I20260925 14:49:39.227716 12472 hybrid_clock.cc:648] HybridClock initialized: now 1790347779227699 us; error 35 us; skew 500 ppm
I20260925 14:49:39.227903 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.227753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54278)
0925 14:49:39.227830 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.227859 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260925 14:49:39.228292 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.228200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54282)
0925 14:49:39.228244 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.228270 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 14:49:39.228381 12465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:35445: Network error: Client connection negotiation failed: client connection to 127.21.205.253:35445: connect: Connection refused (error 111)
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:39.230643 12472 init.cc:377] Logged in from keytab as kudu/127.21.205.253@KRBTEST.COM (short username kudu)
I20260925 14:49:39.231168 12472 webserver.cc:460] Webserver started at http://127.21.205.253:45239/ using document root <none> and password file <none>
I20260925 14:49:39.231415 12472 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:39.231487 12472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:39.231645 12472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:39.232856 12472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data/instance:
uuid: "f31529be377f464e9e6abae34872cc15"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f06e6a4421fdc5f2599c1289fd3f61ca"
server_key_iv: "70a16a4ac4fc2e291b90897fc7ec3c4f"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.233248 12472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal/instance:
uuid: "f31529be377f464e9e6abae34872cc15"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f06e6a4421fdc5f2599c1289fd3f61ca"
server_key_iv: "70a16a4ac4fc2e291b90897fc7ec3c4f"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.234813 12472 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:39.235625 12487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:39.235846 12472 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:39.235950 12472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "f31529be377f464e9e6abae34872cc15"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f06e6a4421fdc5f2599c1289fd3f61ca"
server_key_iv: "70a16a4ac4fc2e291b90897fc7ec3c4f"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.236048 12472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:39.254065 12472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:39.256680 12472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:39.256862 12472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:39.261307 12472 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.253:35445
I20260925 14:49:39.261366 12540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.253:35445 every 8 connection(s)
I20260925 14:49:39.261770 12472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
I20260925 14:49:39.262907 12541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:39.263409 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12472
I20260925 14:49:39.263535 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-1/wal/instance
I20260925 14:49:39.263744 22327 external_mini_cluster.cc:1428] Setting key da44406e0bd7efd873b638a3d7154be0
I20260925 14:49:39.265457 12541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.205.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.252@KRBTEST.COM" created.
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:39.279853 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.266386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42957 (local address 127.0.0.1:49714)
0925 14:49:39.266613 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.266659 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.267156 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.267464 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.267468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.267922 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.268235 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.268242 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.269355 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.269356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.270764 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.270768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.271357 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.271358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.271517 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.272227 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.272238 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.275928 (+ 3690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.277837 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.277849 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.277852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.278226 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.278478 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.278479 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.278479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.278633 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.278978 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.278987 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.279188 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.279305 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.279514 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":130,"threads_started":1}
I20260925 14:49:39.280097 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.266422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.21.205.254:42957)
0925 14:49:39.266493 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.266495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.266756 (+ 261us) server_negotiation.cc:407] Connection header received
0925 14:49:39.267261 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.267263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.267277 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.267360 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.268360 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.269230 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.270884 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.271244 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.271337 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.276080 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.276082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.276083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.276094 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.277733 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.278333 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.278335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.278336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.278406 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.278705 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.278707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.278707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.278883 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.278948 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.279902 (+ 954us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.280005 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 14:49:39.280783 12541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:39.290422 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.281095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54288)
0925 14:49:39.281150 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.281162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.281203 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.281506 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.281508 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.281690 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.281778 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.281780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.283173 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.283174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.284116 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.284119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.284662 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.284663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.284823 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.285282 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.285288 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.286985 (+ 1697us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.289069 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.289071 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.289071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.289340 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.289595 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.289596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.289597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.289656 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.289965 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.289966 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.290043 (+ 77us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.290130 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.290276 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:39.290433 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.281169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54288 (local address 127.21.205.253:35445)
0925 14:49:39.281334 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.281339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.281370 (+ 31us) server_negotiation.cc:407] Connection header received
0925 14:49:39.281392 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.281395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.281408 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.281484 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.281874 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.283062 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.284207 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.284545 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.284607 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.287080 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.287089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.287090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.287097 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.288965 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.289459 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.289463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.289463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.289518 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.289719 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.289720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.289721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.289889 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.289989 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.290167 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.290334 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260925 14:49:39.291282 12541 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
I20260925 14:49:39.291703 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.291549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55282)
0925 14:49:39.291610 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.291666 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 14:49:39.291878 12488 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
I20260925 14:49:39.292295 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:46683
--webserver_interface=127.21.205.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:44787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:49:39.292369 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.292246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55296)
0925 14:49:39.292303 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.292335 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 14:49:39.292701 12541 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:46683: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
I20260925 14:49:39.319543 12465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } attempt: 3
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:39.327131 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.319988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54298)
0925 14:49:39.320057 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.320068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.320135 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.320302 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.320303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.320472 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.320542 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.320544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.321248 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.321249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.321799 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.321801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.322214 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.322215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.322297 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.322802 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.322807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.324082 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.325977 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.325978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.325978 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.326226 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.326509 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.326510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.326510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.326569 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.326861 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.326862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.326908 (+ 46us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.326961 (+ 53us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.327053 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:39.327211 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.320081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.21.205.253:35445)
0925 14:49:39.320156 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.320159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.320169 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:49:39.320199 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.320200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.320212 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.320293 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.320642 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.321150 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.321891 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.322133 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.322217 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.324185 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.324187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.324187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.324197 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.325870 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.326345 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.326347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.326347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.326426 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.326633 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.326634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.326634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.326790 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.326867 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.327014 (+ 147us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.327121 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 14:49:39.327690 12465 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:39.328114 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.328006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55310)
0925 14:49:39.328060 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.328091 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 14:49:39.328454 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.328375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55316)
0925 14:49:39.328415 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.328438 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260925 14:49:39.328562 12465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:46683: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
I20260925 14:49:39.341957 12541 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } attempt: 1
I20260925 14:49:39.342450 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.342312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55326)
0925 14:49:39.342377 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.342418 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 14:49:39.342813 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.342710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55334)
0925 14:49:39.342761 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.342786 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260925 14:49:39.342906 12541 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:46683: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
I20260925 14:49:39.359737 12465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } attempt: 1
I20260925 14:49:39.360287 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.360136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55342)
0925 14:49:39.360208 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.360255 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 14:49:39.360677 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.360576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55346)
0925 14:49:39.360622 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.360646 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 14:49:39.360767 12465 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:46683: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
W20260925 14:49:39.386085 12548 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:39.386409 12548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:39.386446 12548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:39.386499 12548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:39.387928 12548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:39.388015 12548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:39.388043 12548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:39.388087 12548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:39.388115 12548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:39.389948 12548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:44787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:42957,127.21.205.253:35445,127.21.205.252:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.205.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:39.390309 12548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:39.390587 12548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:39.396211 12554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:39.396231 12553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:39.396462 12548 server_base.cc:1034] running on GCE node
W20260925 14:49:39.396211 12556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:39.396773 12548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:39.397114 12548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:39.398311 12548 hybrid_clock.cc:648] HybridClock initialized: now 1790347779398287 us; error 37 us; skew 500 ppm
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:39.401033 12548 init.cc:377] Logged in from keytab as kudu/127.21.205.252@KRBTEST.COM (short username kudu)
I20260925 14:49:39.401464 12548 webserver.cc:460] Webserver started at http://127.21.205.252:43525/ using document root <none> and password file <none>
I20260925 14:49:39.401667 12548 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:39.401710 12548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:39.401813 12548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:39.402899 12548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data/instance:
uuid: "760c5fba733345659a2bd7e7cc9afada"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f58c10c03782ee9cc46565b6d95d018d"
server_key_iv: "c63c411fb21b538ac12cf2142730463e"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.403218 12548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal/instance:
uuid: "760c5fba733345659a2bd7e7cc9afada"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f58c10c03782ee9cc46565b6d95d018d"
server_key_iv: "c63c411fb21b538ac12cf2142730463e"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.404700 12548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:39.405464 12563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:39.405669 12548 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:39.405763 12548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "760c5fba733345659a2bd7e7cc9afada"
format_stamp: "Formatted at 2026-09-25 14:49:39 on dist-test-slave-d7lw"
server_key: "f58c10c03782ee9cc46565b6d95d018d"
server_key_iv: "c63c411fb21b538ac12cf2142730463e"
server_key_version: "encryptionkey@0"
I20260925 14:49:39.405828 12548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:39.411082 12541 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } attempt: 2
I20260925 14:49:39.411550 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.411414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55356)
0925 14:49:39.411479 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.411520 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 14:49:39.411960 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.411851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55364)
0925 14:49:39.411902 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.411929 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260925 14:49:39.412057 12541 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:46683: Network error: Client connection negotiation failed: client connection to 127.21.205.252:46683: connect: Connection refused (error 111)
I20260925 14:49:39.429008 12548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:39.431085 12548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:39.431257 12548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:39.435411 12548 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.252:46683
I20260925 14:49:39.435480 12615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.252:46683 every 8 connection(s)
I20260925 14:49:39.435886 12465 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } attempt: 2
I20260925 14:49:39.435959 12548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
I20260925 14:49:39.436923 12616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:39.437978 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12548
I20260925 14:49:39.438098 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790347486435597-22327-0/minicluster-data/master-2/wal/instance
I20260925 14:49:39.438315 22327 external_mini_cluster.cc:1428] Setting key dfa63aea1da8c4b6ee4f4f9cf3772ba7
I20260925 14:49:39.439193 12616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:39.446866 12467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.436244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55376)
0925 14:49:39.436306 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.436319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.436367 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.437358 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.437360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.437555 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.437599 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.437601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.439135 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.439136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.440049 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.440052 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.440849 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.440850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.440909 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.441414 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.441420 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.442782 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.445218 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.445219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.445220 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.445455 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.445704 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.445704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.445705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.445818 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.446396 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.446397 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.446453 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.446529 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.446630 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 14:49:39.447072 12618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.436378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.21.205.252:46683)
0925 14:49:39.436573 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.436584 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.436616 (+ 32us) server_negotiation.cc:407] Connection header received
0925 14:49:39.437141 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.437150 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.437212 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.437375 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.437711 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.439024 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.440132 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.440774 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.440878 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.442885 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.442891 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.442892 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.442906 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.445076 (+ 2170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.445542 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.445546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.445547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.445643 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.445904 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.445905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.445906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.446300 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.446401 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.446632 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.446766 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":119,"threads_started":1}
I20260925 14:49:39.448931 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.439956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.21.205.254:42957)
0925 14:49:39.440003 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.440005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.440192 (+ 187us) server_negotiation.cc:407] Connection header received
0925 14:49:39.440266 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.440267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.440279 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.440330 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.440797 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.441443 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.442547 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.442832 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.442870 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.445691 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.445692 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.445693 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.445700 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.447432 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.448004 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.448006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.448006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.448064 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.448298 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.448299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.448300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.448479 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.448554 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.448763 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.448844 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:49:39.448983 12621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.439938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:42957 (local address 127.0.0.1:49716)
0925 14:49:39.440074 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.440114 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.440212 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.440347 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.440350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.440633 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.440697 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.440698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.441534 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.441535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.442450 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.442452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.442945 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.442946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.443042 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.443605 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.443609 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.445604 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.447532 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.447541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.447543 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.447925 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.448150 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.448151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.448151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.448217 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.448556 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.448564 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.448622 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.448720 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.448858 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":86,"threads_started":1}
I20260925 14:49:39.449146 12465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: Bootstrap starting.
I20260925 14:49:39.449447 12616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:39.449934 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.438882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.21.205.252:46683)
0925 14:49:39.439174 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.439176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.439182 (+ 6us) server_negotiation.cc:407] Connection header received
0925 14:49:39.439199 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.439200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.439209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.439292 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.440132 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.440877 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.441962 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.442287 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.442351 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.445137 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.445138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.445138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.445146 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.448367 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.448779 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.448780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.448780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.448838 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.449031 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.449032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.449032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.449162 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.449303 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.449699 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.449790 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":89,"threads_started":1}
I20260925 14:49:39.450284 12465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:39.450631 12465 log.cc:824] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:39.451530 12465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: No bootstrap required, opened a new log
I20260925 14:49:39.453744 12465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.453913 12465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ff77b49e2444eb0a6a04ecc1e3ed686, State: Initialized, Role: FOLLOWER
I20260925 14:49:39.454052 12465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.454442 12465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:39.456039 12624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.456220 12624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: This master's current role is: FOLLOWER
W20260925 14:49:39.457017 12636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:39.457131 12636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:39.459889 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.450666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.21.205.254:42957)
0925 14:49:39.450706 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.450708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.450715 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:49:39.450759 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.450760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.450769 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.450823 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.451401 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.452018 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.452865 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.453146 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.453217 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.456396 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.456398 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.456398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.456405 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.458389 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.458838 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.458840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.458840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.458901 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.459113 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.459114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.459114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.459260 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.459399 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.459638 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.459725 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:39.460611 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.449914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54306 (local address 127.21.205.253:35445)
0925 14:49:39.449957 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.449959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.449968 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:39.449995 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.449997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.450009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.450072 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.450748 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.451487 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.452477 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.452814 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.452879 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.456368 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.456370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.456370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.456379 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.459207 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.459641 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.459642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.459643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.459704 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.460030 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.460031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.460031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.460169 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.460214 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.460444 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.460526 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:39.457430 12397 master_runner.cc:186] Error getting master registration for 127.21.205.253:35445: OK, instance_id { permanent_uuid: "f31529be377f464e9e6abae34872cc15" instance_seqno: 1790347779260001 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:49:39.460898 12397 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 14:49:39.461282 12622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.449714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:35445 (local address 127.0.0.1:54306)
0925 14:49:39.449879 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.449891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.449941 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.450273 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.450274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.450456 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.450547 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.450550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.451594 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.451594 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.452384 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.452387 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.452945 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.452946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.453000 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.453473 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.453478 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.455717 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.459321 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.459323 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.459323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.459553 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.459825 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.459826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.459827 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.459886 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.460282 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.460283 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.460333 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.460929 (+ 596us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.461143 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":102,"threads_started":1}
I20260925 14:49:39.461624 12616 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:49:39.469852 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.461956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55384 (local address 127.21.205.252:46683)
0925 14:49:39.461994 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.461996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.462004 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:39.462048 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.462049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.462059 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.462120 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.462422 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.463153 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.464186 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.464539 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.464626 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.466805 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.466806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.466807 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.466814 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.468584 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.468977 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.468978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.468979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.469033 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.469224 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.469225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.469225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.469404 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.469480 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.469635 (+ 155us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.469737 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260925 14:49:39.469921 12622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.461910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55384)
0925 14:49:39.461951 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.461960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.461993 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.462126 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.462126 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.462282 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.462339 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.462341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.463245 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.463246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.464074 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.464077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.464634 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.464635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.464707 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.465150 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.465155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.466645 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.468702 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.468704 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.468704 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.468893 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.469090 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.469091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.469091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.469158 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.469482 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.469484 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.469524 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.469614 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.469802 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":90}
I20260925 14:49:39.470089 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.460978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54310 (local address 127.21.205.253:35445)
0925 14:49:39.461027 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.461029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.461104 (+ 75us) server_negotiation.cc:407] Connection header received
0925 14:49:39.461132 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.461133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.461144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.461207 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.461830 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.462535 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.463220 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.463454 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.463545 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.466793 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.466794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.466795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.466803 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.468591 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.469026 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.469027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.469027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.469084 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.469278 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.469280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.469280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.469425 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.469599 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.469826 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.469914 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:39.471701 12616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: Bootstrap starting.
I20260925 14:49:39.472294 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:39.472568 12616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:39.472869 12616 log.cc:824] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:39.473378 12439 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49726
I20260925 14:49:39.473577 12439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 14:49:39.473635 12439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:39.473791 12616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: No bootstrap required, opened a new log
I20260925 14:49:39.474584 12439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.474861 12439 leader_election.cc:288] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [CANDIDATE]: Term 1 election: Requested vote from peers f31529be377f464e9e6abae34872cc15 (127.21.205.253:35445), 760c5fba733345659a2bd7e7cc9afada (127.21.205.252:46683)
I20260925 14:49:39.476758 12616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.477049 12616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760c5fba733345659a2bd7e7cc9afada, State: Initialized, Role: FOLLOWER
I20260925 14:49:39.477216 12616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.477641 12616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:39.477677 12590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "760c5fba733345659a2bd7e7cc9afada"
I20260925 14:49:39.477792 12590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:39.478820 12590 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff77b49e2444eb0a6a04ecc1e3ed686 in term 1.
I20260925 14:49:39.479099 12413 leader_election.cc:302] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ff77b49e2444eb0a6a04ecc1e3ed686, 760c5fba733345659a2bd7e7cc9afada; no voters:
I20260925 14:49:39.479573 12624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:39.479750 12624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [term 1 LEADER]: Becoming Leader. State: Replica: 3ff77b49e2444eb0a6a04ecc1e3ed686, State: Running, Role: LEADER
I20260925 14:49:39.479897 12624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
W20260925 14:49:39.480268 12657 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:39.480365 12657 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:39.480425 12624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff77b49e2444eb0a6a04ecc1e3ed686. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.480496 12624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:39.481838 12548 master_runner.cc:186] Error getting master registration for 127.21.205.253:35445: OK, instance_id { permanent_uuid: "f31529be377f464e9e6abae34872cc15" instance_seqno: 1790347779260001 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:49:39.484432 12548 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 14:49:39.484915 12646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.485086 12646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:39.488253 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.476360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49736 (local address 127.21.205.254:42957)
0925 14:49:39.476406 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.476408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.477192 (+ 784us) server_negotiation.cc:407] Connection header received
0925 14:49:39.477362 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.477363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.477388 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.477421 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.481257 (+ 3836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.481917 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.482910 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.483196 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.483251 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.485038 (+ 1787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.485039 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.485039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.485047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.486781 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.487234 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.487235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.487236 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.487319 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.487581 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.487581 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.487582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.487745 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.487899 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.488050 (+ 151us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.488135 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:49:39.494544 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.476801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55392 (local address 127.21.205.252:46683)
0925 14:49:39.476847 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.476849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.484048 (+ 7199us) server_negotiation.cc:407] Connection header received
0925 14:49:39.484365 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.484366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.484390 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.484422 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.487889 (+ 3467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.488587 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.489636 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.489949 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.490017 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.491487 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.491488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.491489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.491497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.493157 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.493536 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.493537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.493537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.493593 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.493800 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.493801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.493801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.493947 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.494054 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.494214 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.494307 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:39.496018 12661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:39.496241 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.476343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54326 (local address 127.21.205.253:35445)
0925 14:49:39.476385 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.476387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.476812 (+ 425us) server_negotiation.cc:407] Connection header received
0925 14:49:39.476949 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.476950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.476962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.476993 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.478192 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.478962 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.480649 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.482473 (+ 1824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.482527 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.489686 (+ 7159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.489689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.489689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.489700 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.491577 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.492025 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.492027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.492027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.492091 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.492352 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.492353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.492353 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.492537 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.492642 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.496049 (+ 3407us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.496150 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260925 14:49:39.496623 12661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:39.498461 12590 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [term 1 FOLLOWER]: Refusing update from remote peer 3ff77b49e2444eb0a6a04ecc1e3ed686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:39.500501 12659 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:39.502219 12646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff77b49e2444eb0a6a04ecc1e3ed686. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.502355 12646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:39.503183 12541 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } attempt: 3
I20260925 14:49:39.503389 12624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.503482 12624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:39.503757 12662 mvcc.cc:205] Tried to move back new op lower bound from 7333264504822865920 to 7333264504750657536. Current Snapshot: MvccSnapshot[applied={T|T < 7333264504822865920}]
I20260925 14:49:39.504120 12646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.504212 12646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada [sys.catalog]: This master's current role is: FOLLOWER
Sep 25 14:49:39 dist-test-slave-d7lw krb5kdc[12384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347779, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:49:39.510221 12661 catalog_manager.cc:1347] Generated new cluster ID: f44f3c9ef2e94c18aced44b1d473e9ed
I20260925 14:49:39.510404 12661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:39.512876 12544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.503480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:46683 (local address 127.0.0.1:55394)
0925 14:49:39.503571 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:39.503583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:39.503637 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:39.503916 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:39.503917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:39.504108 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:39.504166 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.504168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.505097 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.505098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.505922 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:39.505924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.506376 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.506377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:39.506447 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.506955 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:39.506960 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:39.508623 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:39.511697 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.511698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.511699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.511966 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.512175 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:39.512176 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:39.512176 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:39.512240 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:39.512542 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:39.512543 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:39.512594 (+ 51us) client_negotiation.cc:769] Sending connection context
0925 14:49:39.512656 (+ 62us) client_negotiation.cc:240] Negotiation successful
0925 14:49:39.512770 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260925 14:49:39.513247 12620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:39.503713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55394 (local address 127.21.205.252:46683)
0925 14:49:39.503753 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.503755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.503762 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:49:39.503804 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.503805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.503815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.503864 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.504252 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.505020 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.506007 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:39.506302 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:39.506371 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:39.508745 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:39.508747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:39.508748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:39.508758 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:39.510652 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.512049 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.512051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.512051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.512112 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:39.512322 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:39.512323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:39.512323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:39.512460 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:39.512518 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:39.513071 (+ 553us) server_negotiation.cc:299] Negotiation successful
0925 14:49:39.513151 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:39.514926 12541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: Bootstrap starting.
I20260925 14:49:39.515744 12541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:39.516111 12541 log.cc:824] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:39.520360 12663 mvcc.cc:205] Tried to move back new op lower bound from 7333264504822865920 to 7333264504750657536. Current Snapshot: MvccSnapshot[applied={T|T < 7333264504822865920}]
I20260925 14:49:39.524652 12541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: No bootstrap required, opened a new log
I20260925 14:49:39.526870 12541 raft_consensus.cc:348] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.527055 12541 raft_consensus.cc:729] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31529be377f464e9e6abae34872cc15, State: Initialized, Role: FOLLOWER
I20260925 14:49:39.527166 12541 consensus_queue.cc:260] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } }
I20260925 14:49:39.527607 12515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f31529be377f464e9e6abae34872cc15"
I20260925 14:49:39.527670 12514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:39.528878 12514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [term 1 FOLLOWER]: Refusing update from remote peer 3ff77b49e2444eb0a6a04ecc1e3ed686: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:39.529209 12624 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:39.530236 12515 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ff77b49e2444eb0a6a04ecc1e3ed686 in term 1.
I20260925 14:49:39.527583 12541 sys_catalog.cc:564] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:39.527552 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.533803 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:39.533911 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ff77b49e2444eb0a6a04ecc1e3ed686. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.534040 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:39.534099 12668 sys_catalog.cc:455] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.534176 12668 sys_catalog.cc:458] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:39.540400 12670 mvcc.cc:205] Tried to move back new op lower bound from 7333264504822865920 to 7333264504750657536. Current Snapshot: MvccSnapshot[applied={T|T < 7333264504822865920}]
I20260925 14:49:39.541945 12683 catalog_manager.cc:1259] Loaded cluster ID: f44f3c9ef2e94c18aced44b1d473e9ed
I20260925 14:49:39.542002 12683 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: loading cluster ID for follower catalog manager: success
W20260925 14:49:39.542097 12683 catalog_manager.cc:1576] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260925 14:49:39.542141 12683 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:39.556576 12659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3ff77b49e2444eb0a6a04ecc1e3ed686" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 42957 } } peers { permanent_uuid: "f31529be377f464e9e6abae34872cc15" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 35445 } } peers { permanent_uuid: "760c5fba733345659a2bd7e7cc9afada" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 46683 } } }
I20260925 14:49:39.556727 12659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:39.556733 12661 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:39.557474 12661 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:39.575863 12661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ff77b49e2444eb0a6a04ecc1e3ed686: Generated new TSK 0
I20260925 14:49:39.576156 12661 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:49:39.604341 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.603208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.21.205.253:35445)
0925 14:49:39.603267 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.603270 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.603590 (+ 320us) server_negotiation.cc:407] Connection header received
0925 14:49:39.603701 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.603702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.603713 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.603744 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.604262 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54336: BlockingRecv error: recv got EOF from 127.0.0.1:54336 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260925 14:49:39.605788 12468 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.604878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49748 (local address 127.21.205.254:42957)
0925 14:49:39.604919 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.604921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.605206 (+ 285us) server_negotiation.cc:407] Connection header received
0925 14:49:39.605322 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.605324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.605335 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.605372 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.605714 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49748: BlockingRecv error: recv got EOF from 127.0.0.1:49748 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 14:49:39.606717 12620 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.605095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55400 (local address 127.21.205.252:46683)
0925 14:49:39.605135 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.605136 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.606219 (+ 1083us) server_negotiation.cc:407] Connection header received
0925 14:49:39.606325 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.606326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.606339 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.606368 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.606659 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55400: BlockingRecv error: recv got EOF from 127.0.0.1:55400 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 14:49:39.609354 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.608418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.21.205.253:35445)
0925 14:49:39.608469 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.608471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.608726 (+ 255us) server_negotiation.cc:407] Connection header received
0925 14:49:39.608857 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.608858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.608879 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.608909 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.609298 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54348: BlockingRecv error: recv got EOF from 127.0.0.1:54348 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260925 14:49:39.612685 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.611755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54356 (local address 127.21.205.253:35445)
0925 14:49:39.611818 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.611820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.612097 (+ 277us) server_negotiation.cc:407] Connection header received
0925 14:49:39.612207 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.612208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.612220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.612259 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.612627 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54356: BlockingRecv error: recv got EOF from 127.0.0.1:54356 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 14:49:39.616902 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.616026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.21.205.253:35445)
0925 14:49:39.616072 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.616074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.616346 (+ 272us) server_negotiation.cc:407] Connection header received
0925 14:49:39.616449 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.616450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.616461 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.616492 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.616844 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54372: BlockingRecv error: recv got EOF from 127.0.0.1:54372 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260925 14:49:39.623283 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.622365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54384 (local address 127.21.205.253:35445)
0925 14:49:39.622408 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.622410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.622678 (+ 268us) server_negotiation.cc:407] Connection header received
0925 14:49:39.622784 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.622786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.622797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.622829 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.623209 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54384: BlockingRecv error: recv got EOF from 127.0.0.1:54384 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260925 14:49:39.633741 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.632788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54394 (local address 127.21.205.253:35445)
0925 14:49:39.632833 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.632835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.633126 (+ 291us) server_negotiation.cc:407] Connection header received
0925 14:49:39.633244 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.633246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.633257 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.633289 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.633676 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54394: BlockingRecv error: recv got EOF from 127.0.0.1:54394 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 14:49:39.652304 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.651270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.21.205.253:35445)
0925 14:49:39.651317 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.651320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.651621 (+ 301us) server_negotiation.cc:407] Connection header received
0925 14:49:39.651734 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.651735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.651749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.651807 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.652212 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54396: BlockingRecv error: recv got EOF from 127.0.0.1:54396 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 14:49:39.686674 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.685893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.21.205.253:35445)
0925 14:49:39.685944 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.685946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.686061 (+ 115us) server_negotiation.cc:407] Connection header received
0925 14:49:39.686121 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.686122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.686134 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.686212 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.686612 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54410: BlockingRecv error: recv got EOF from 127.0.0.1:54410 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260925 14:49:39.752791 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.751974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54418 (local address 127.21.205.253:35445)
0925 14:49:39.752032 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.752036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.752177 (+ 141us) server_negotiation.cc:407] Connection header received
0925 14:49:39.752308 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.752309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.752321 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.752373 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.752723 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54418: BlockingRecv error: recv got EOF from 127.0.0.1:54418 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260925 14:49:39.882632 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:39.881834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.21.205.253:35445)
0925 14:49:39.881912 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:39.881915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:39.882019 (+ 104us) server_negotiation.cc:407] Connection header received
0925 14:49:39.882122 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:39.882123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:39.882135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:39.882210 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:39.882562 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54434: BlockingRecv error: recv got EOF from 127.0.0.1:54434 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 14:49:40.140743 12546 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:40.139940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.21.205.253:35445)
0925 14:49:40.140010 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.140013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.140075 (+ 62us) server_negotiation.cc:407] Connection header received
0925 14:49:40.140188 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.140189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.140201 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.140276 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:40.140645 (+ 369us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54448: BlockingRecv error: recv got EOF from 127.0.0.1:54448 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 14:49:40.485023 12657 catalog_manager.cc:1259] Loaded cluster ID: f44f3c9ef2e94c18aced44b1d473e9ed
I20260925 14:49:40.485136 12657 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: loading cluster ID for follower catalog manager: success
I20260925 14:49:40.485952 12657 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: acquiring CA information for follower catalog manager: success
I20260925 14:49:40.486287 12657 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 760c5fba733345659a2bd7e7cc9afada: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:49:40.543334 12683 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: acquiring CA information for follower catalog manager: success
I20260925 14:49:40.543776 12683 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f31529be377f464e9e6abae34872cc15: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:49:40.657156 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:40.654029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54464 (local address 127.21.205.253:35445)
0925 14:49:40.654158 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.654160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.654259 (+ 99us) server_negotiation.cc:407] Connection header received
0925 14:49:40.654365 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.654366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.654451 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.654544 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:40.654714 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.655388 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.656211 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.656499 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.656579 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:40.656765 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:40.656895 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:40.656985 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:40.657025 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 14:49:40.657048 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260925 14:49:40.661528 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:40.659086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55416 (local address 127.21.205.252:46683)
0925 14:49:40.659190 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.659192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.659201 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:40.659268 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.659269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.659304 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.659379 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:40.659524 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.660075 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.660805 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.661007 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.661057 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:40.661191 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:40.661308 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:40.661364 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:40.661442 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 14:49:40.661460 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"thread_start_us":52,"threads_started":1}
I20260925 14:49:40.666460 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:40.663428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.21.205.253:35445)
0925 14:49:40.663473 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.663475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.663642 (+ 167us) server_negotiation.cc:407] Connection header received
0925 14:49:40.663691 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.663692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.663704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.663768 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:40.663946 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.664589 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.665575 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.665903 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.665959 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:40.666145 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:40.666181 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:40.666273 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:40.666335 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 14:49:40.666358 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260925 14:49:40.666668 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:40.663756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55420 (local address 127.21.205.252:46683)
0925 14:49:40.663820 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.663822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.663911 (+ 89us) server_negotiation.cc:407] Connection header received
0925 14:49:40.663992 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.663993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.664003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.664057 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:40.664158 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.664871 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.665846 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.666153 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.666211 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:40.666389 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:40.666420 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:40.666447 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:40.666549 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 14:49:40.666576 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 14:49:40.666852 12761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:40.663446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49754 (local address 127.21.205.254:42957)
0925 14:49:40.663733 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:40.663736 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:40.663902 (+ 166us) server_negotiation.cc:407] Connection header received
0925 14:49:40.664012 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:40.664013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:40.664068 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:40.664124 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:40.664336 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.665059 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.665795 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:40.666092 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:40.666161 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:40.666425 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:40.666548 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:40.666625 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:40.666735 (+ 110us) server_negotiation.cc:299] Negotiation successful
0925 14:49:40.666776 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":61,"threads_started":1}
I20260925 14:49:40.668771 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12397
I20260925 14:49:40.674692 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12472
I20260925 14:49:40.680415 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12548
2026-09-25T14:49:40Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2870 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 14:49:40 dist-test-slave-d7lw krb5kdc[12766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 14:49:40 dist-test-slave-d7lw krb5kdc[12766](info): set up 2 sockets
Sep 25 14:49:40 dist-test-slave-d7lw krb5kdc[12766](info): commencing operation
krb5kdc: starting...
W20260925 14:49:41.726963 22327 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/test-user.keytab.
Sep 25 14:49:41 dist-test-slave-d7lw krb5kdc[12766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T14:49:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T14:49:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.205.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
Entry for principal kudu/127.21.205.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab.
I20260925 14:49:41.844475 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:39057
--webserver_interface=127.21.205.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 14:49:41.925940 12780 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:41.926143 12780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:41.926160 12780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:41.926182 12780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:41.927245 12780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:41.927279 12780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:41.927294 12780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:41.927306 12780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:41.927318 12780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:41.928828 12780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.254.keytab
--principal=kudu/127.21.205.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.254:39057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.205.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:41.929014 12780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:41.929177 12780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:41.934125 12786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:41.934154 12788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:41.934253 12780 server_base.cc:1034] running on GCE node
W20260925 14:49:41.934137 12785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:41.934569 12780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:41.934870 12780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:41.936030 12780 hybrid_clock.cc:648] HybridClock initialized: now 1790347781936014 us; error 33 us; skew 500 ppm
Sep 25 14:49:41 dist-test-slave-d7lw krb5kdc[12766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:41.942471 12780 init.cc:377] Logged in from keytab as kudu/127.21.205.254@KRBTEST.COM (short username kudu)
I20260925 14:49:41.942962 12780 webserver.cc:460] Webserver started at http://127.21.205.254:39927/ using document root <none> and password file <none>
I20260925 14:49:41.943207 12780 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:41.943259 12780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:41.943409 12780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:41.944593 12780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data/instance:
uuid: "c82c0b5a9e404f36a5609303602b9be7"
format_stamp: "Formatted at 2026-09-25 14:49:41 on dist-test-slave-d7lw"
server_key: "47f57c05525f7addf0062f3399e0aa98"
server_key_iv: "90a75a22b417ee6e64410c2ced77dd25"
server_key_version: "encryptionkey@0"
I20260925 14:49:41.944991 12780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance:
uuid: "c82c0b5a9e404f36a5609303602b9be7"
format_stamp: "Formatted at 2026-09-25 14:49:41 on dist-test-slave-d7lw"
server_key: "47f57c05525f7addf0062f3399e0aa98"
server_key_iv: "90a75a22b417ee6e64410c2ced77dd25"
server_key_version: "encryptionkey@0"
I20260925 14:49:41.946333 12780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260925 14:49:41.947032 12795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:41.947223 12780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:41.947316 12780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal
uuid: "c82c0b5a9e404f36a5609303602b9be7"
format_stamp: "Formatted at 2026-09-25 14:49:41 on dist-test-slave-d7lw"
server_key: "47f57c05525f7addf0062f3399e0aa98"
server_key_iv: "90a75a22b417ee6e64410c2ced77dd25"
server_key_version: "encryptionkey@0"
I20260925 14:49:41.947371 12780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:41.958557 12780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:41.960778 12780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:41.960882 12780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:41.964598 12780 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.254:39057
I20260925 14:49:41.964650 12847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.254:39057 every 8 connection(s)
I20260925 14:49:41.965000 12780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/data/info.pb
I20260925 14:49:41.965875 12848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:41.967391 12848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:41.969146 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12780
I20260925 14:49:41.969276 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-0/wal/instance
I20260925 14:49:41.969460 22327 external_mini_cluster.cc:1428] Setting key 6ddf562f787550f7da2c0519b3ca80b2
Sep 25 14:49:41 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.21.205.253@KRBTEST.COM; defaulting to no policy
I20260925 14:49:41.977033 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:41.968069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39057 (local address 127.0.0.1:39080)
0925 14:49:41.968288 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:41.968316 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:41.968758 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:41.968994 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:41.968996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:41.969215 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:41.969453 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:41.969457 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:41.970449 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:41.970450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:41.971084 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:41.971086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:41.971581 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:41.971581 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:41.971659 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:41.972142 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:41.972152 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:41.973811 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:41.975503 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:41.975509 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:41.975511 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:41.975697 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:41.976011 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:41.976012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:41.976013 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:41.976101 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:41.976457 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:41.976462 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:41.976524 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 14:49:41.976631 (+ 107us) client_negotiation.cc:240] Negotiation successful
0925 14:49:41.976862 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":97,"thread_start_us":119,"threads_started":1}
I20260925 14:49:41.977035 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:41.968102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.21.205.254:39057)
0925 14:49:41.968297 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:41.968306 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:41.968393 (+ 87us) server_negotiation.cc:407] Connection header received
0925 14:49:41.968875 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:41.968877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:41.968892 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:41.968990 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:41.969555 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:41.970340 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:41.971180 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:41.971486 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:41.971607 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:41.973903 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:41.973909 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:41.973910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:41.973916 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:41.975404 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:41.975864 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:41.975867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:41.975867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:41.975937 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:41.976167 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:41.976168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:41.976169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:41.976383 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:41.976476 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:41.976665 (+ 189us) server_negotiation.cc:299] Negotiation successful
0925 14:49:41.976800 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260925 14:49:41.977743 12848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:41.978099 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:41.977981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47842)
0925 14:49:41.978038 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:41.978080 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 14:49:41.978241 12799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
I20260925 14:49:41.978646 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:41.978564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47854)
0925 14:49:41.978608 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:41.978631 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 14:49:41.978773 12848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33645: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
Entry for principal kudu/127.21.205.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab.
I20260925 14:49:41.994395 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:33645
--webserver_interface=127.21.205.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:49:42.028020 12848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } attempt: 1
I20260925 14:49:42.028535 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.028379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47870)
0925 14:49:42.028448 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.028498 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 14:49:42.028940 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.028828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47882)
0925 14:49:42.028871 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.028918 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260925 14:49:42.029040 12848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33645: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
W20260925 14:49:42.075546 12855 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:42.075765 12855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:42.075860 12855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:42.075902 12855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:42.077054 12855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:42.077090 12855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:42.077104 12855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:42.077117 12855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:42.077129 12855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:42.078583 12855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.253.keytab
--principal=kudu/127.21.205.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.253:33645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.205.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:42.078778 12855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:42.078953 12855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 14:49:42.083986 12855 server_base.cc:1034] running on GCE node
W20260925 14:49:42.083937 12860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:42.083930 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:42.083952 12861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:42.084429 12855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:42.084771 12855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:42.085924 12855 hybrid_clock.cc:648] HybridClock initialized: now 1790347782085904 us; error 36 us; skew 500 ppm
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:42.088521 12855 init.cc:377] Logged in from keytab as kudu/127.21.205.253@KRBTEST.COM (short username kudu)
I20260925 14:49:42.088982 12855 webserver.cc:460] Webserver started at http://127.21.205.253:38309/ using document root <none> and password file <none>
I20260925 14:49:42.089219 12855 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:42.089264 12855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:42.089425 12855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:42.090518 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data/instance:
uuid: "a38ad9a475144c61a38ffa188889c711"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "9c1f8a87be93f54d31c3f63e625d0c82"
server_key_iv: "a1e37e8f3c7851c9987bf0165824608c"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.090898 12855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal/instance:
uuid: "a38ad9a475144c61a38ffa188889c711"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "9c1f8a87be93f54d31c3f63e625d0c82"
server_key_iv: "a1e37e8f3c7851c9987bf0165824608c"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.092329 12855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:42.093036 12870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:42.093206 12855 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260925 14:49:42.093297 12855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal
uuid: "a38ad9a475144c61a38ffa188889c711"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "9c1f8a87be93f54d31c3f63e625d0c82"
server_key_iv: "a1e37e8f3c7851c9987bf0165824608c"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.093353 12855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:42.097186 12848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } attempt: 2
I20260925 14:49:42.097631 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.097493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47886)
0925 14:49:42.097555 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.097596 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:42.098011 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.097933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47898)
0925 14:49:42.097972 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.097996 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 14:49:42.098134 12848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.253:33645: Network error: Client connection negotiation failed: client connection to 127.21.205.253:33645: connect: Connection refused (error 111)
I20260925 14:49:42.105865 12855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:42.107725 12855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:42.107911 12855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:42.111738 12855 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.253:33645
I20260925 14:49:42.111865 12922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.253:33645 every 8 connection(s)
I20260925 14:49:42.112177 12855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/data/info.pb
I20260925 14:49:42.113085 12923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:42.114634 12923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.119221 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12855
I20260925 14:49:42.119299 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-1/wal/instance
I20260925 14:49:42.119477 22327 external_mini_cluster.cc:1428] Setting key b635a0ad94b9df671be9dc14487726a8
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:42.124024 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.115336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39090 (local address 127.21.205.254:39057)
0925 14:49:42.115402 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.115403 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.115617 (+ 214us) server_negotiation.cc:407] Connection header received
0925 14:49:42.116126 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.116127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.116139 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.116197 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.116795 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.117264 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.118185 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.118374 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.118422 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.120944 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.120946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.120946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.120954 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.122489 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.122978 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.122979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.122980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.123028 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.123317 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.123318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.123318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.123437 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.123510 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.123895 (+ 385us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.123961 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 14:49:42.124338 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.115300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39057 (local address 127.0.0.1:39090)
0925 14:49:42.115486 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.115526 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.116042 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.116272 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.116276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.116515 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.116722 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.116726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.117340 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.117340 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.118115 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.118117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.118445 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.118446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.118558 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.119062 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.119070 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.120833 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.122574 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.122581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.122584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.122903 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.123093 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.123094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.123094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.123252 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.123515 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.123522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.123703 (+ 181us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.123870 (+ 167us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.124051 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260925 14:49:42.124954 12923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.205.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.205.252@KRBTEST.COM" created.
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:42.134222 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.125224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47914)
0925 14:49:42.125303 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.125313 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.125358 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.125626 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.125627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.125760 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.125833 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.125834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.127204 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.127205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.127967 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.127970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.128619 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.128620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.128686 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.129064 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.129069 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.131331 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.133167 (+ 1836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.133168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.133168 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.133330 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.133549 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.133550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.133550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.133604 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.133888 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.133889 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.133926 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.133986 (+ 60us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.134087 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 14:49:42.134254 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.125289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47914 (local address 127.21.205.253:33645)
0925 14:49:42.125460 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.125465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.125491 (+ 26us) server_negotiation.cc:407] Connection header received
0925 14:49:42.125517 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.125519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.125535 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.125620 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.125919 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.127095 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.128060 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.128542 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.128603 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.131432 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.131440 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.131441 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.131447 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.133042 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.133423 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.133426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.133426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.133483 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.133674 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.133675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.133676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.133817 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.133909 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.134032 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.134140 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260925 14:49:42.134996 12923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.135355 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.135235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36044)
0925 14:49:42.135282 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.135324 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 14:49:42.135510 12874 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
I20260925 14:49:42.135936 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.135861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36056)
0925 14:49:42.135891 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.135915 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260925 14:49:42.136045 12923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:40459: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
Entry for principal kudu/127.21.205.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab.
I20260925 14:49:42.146114 22327 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskY8Adyw/build/release/bin/kudu
/tmp/dist-test-taskY8Adyw/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:40459
--webserver_interface=127.21.205.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.205.212:36917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 14:49:42.185293 12923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } attempt: 1
I20260925 14:49:42.185801 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.185661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36062)
0925 14:49:42.185729 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.185772 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 14:49:42.186262 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.186169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36078)
0925 14:49:42.186210 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.186245 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260925 14:49:42.186374 12923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:40459: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
I20260925 14:49:42.189258 12848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } attempt: 3
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:42.196916 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.189524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47924)
0925 14:49:42.189582 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.189593 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.189644 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.189876 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.189876 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.190016 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.190095 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.190097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.191082 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.191083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.191616 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.191617 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.192186 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.192187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.192249 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.192727 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.192731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.193917 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.195829 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.195830 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.195831 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.196003 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.196218 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.196218 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.196219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.196276 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.196583 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.196583 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.196641 (+ 58us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.196696 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.196842 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260925 14:49:42.196933 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.189617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47924 (local address 127.21.205.253:33645)
0925 14:49:42.189684 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.189686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.189700 (+ 14us) server_negotiation.cc:407] Connection header received
0925 14:49:42.189759 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.189761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.189778 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.189871 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.190179 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.190980 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.191708 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.192105 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.192189 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.194022 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.194024 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.194024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.194032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.195707 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.196075 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.196076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.196077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.196145 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.196340 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.196341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.196341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.196480 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.196557 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.196748 (+ 191us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.196834 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 14:49:42.197328 12848 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.197657 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.197549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36094)
0925 14:49:42.197599 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.197626 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260925 14:49:42.197973 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.197917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36106)
0925 14:49:42.197940 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.197961 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260925 14:49:42.198078 12848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:40459: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
I20260925 14:49:42.229216 12848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } attempt: 1
I20260925 14:49:42.229815 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.229634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36114)
0925 14:49:42.229725 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.229781 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 14:49:42.230221 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.230110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36128)
0925 14:49:42.230155 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.230187 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260925 14:49:42.230345 12848 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:40459: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
W20260925 14:49:42.231745 12930 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 14:49:42.232038 12930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 14:49:42.232059 12930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 14:49:42.232089 12930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 14:49:42.233245 12930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 14:49:42.233290 12930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 14:49:42.233309 12930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 14:49:42.233322 12930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 14:49:42.233336 12930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 14:49:42.234855 12930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.205.212:36917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.205.254:39057,127.21.205.253:33645,127.21.205.252:40459
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/krb5kdc/kudu_127.21.205.252.keytab
--principal=kudu/127.21.205.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.205.252:40459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.205.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 25 Sep 2026 13:55:42 UTC on ccycloud.quasar-zoybvp.root.comops.site
build id 4672723
I20260925 14:49:42.235082 12930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 14:49:42.235277 12930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 14:49:42.240634 12936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:42.240691 12930 server_base.cc:1034] running on GCE node
W20260925 14:49:42.240677 12935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 14:49:42.240653 12938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 14:49:42.241122 12930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 14:49:42.241436 12930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 14:49:42.242625 12930 hybrid_clock.cc:648] HybridClock initialized: now 1790347782242608 us; error 43 us; skew 500 ppm
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 14:49:42.245206 12930 init.cc:377] Logged in from keytab as kudu/127.21.205.252@KRBTEST.COM (short username kudu)
I20260925 14:49:42.245601 12930 webserver.cc:460] Webserver started at http://127.21.205.252:37219/ using document root <none> and password file <none>
I20260925 14:49:42.245846 12930 fs_manager.cc:359] Metadata directory not provided
I20260925 14:49:42.245893 12930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 14:49:42.246042 12930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 14:49:42.247128 12930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data/instance:
uuid: "5889780c6887422a820e98a5c2cd7c0a"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "76b4970534239f14c576464787b96635"
server_key_iv: "165cc87b8161ad464244f41744b79f48"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.247509 12930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal/instance:
uuid: "5889780c6887422a820e98a5c2cd7c0a"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "76b4970534239f14c576464787b96635"
server_key_iv: "165cc87b8161ad464244f41744b79f48"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.248989 12930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260925 14:49:42.249787 12945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 14:49:42.250017 12930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 14:49:42.250120 12930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data,/tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal
uuid: "5889780c6887422a820e98a5c2cd7c0a"
format_stamp: "Formatted at 2026-09-25 14:49:42 on dist-test-slave-d7lw"
server_key: "76b4970534239f14c576464787b96635"
server_key_iv: "165cc87b8161ad464244f41744b79f48"
server_key_version: "encryptionkey@0"
I20260925 14:49:42.250205 12930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 14:49:42.254518 12923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } attempt: 2
I20260925 14:49:42.254952 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.254827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36134)
0925 14:49:42.254883 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.254922 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 14:49:42.255275 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.255196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36146)
0925 14:49:42.255233 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.255258 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260925 14:49:42.255404 12923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.205.252:40459: Network error: Client connection negotiation failed: client connection to 127.21.205.252:40459: connect: Connection refused (error 111)
I20260925 14:49:42.277746 12930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 14:49:42.280179 12930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 14:49:42.280339 12930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 14:49:42.284428 12930 rpc_server.cc:307] RPC server started. Bound to: 127.21.205.252:40459
I20260925 14:49:42.284482 12999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.205.252:40459 every 8 connection(s)
I20260925 14:49:42.284852 12930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/data/info.pb
I20260925 14:49:42.285713 13000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 14:49:42.287287 13000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.291244 22327 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskY8Adyw/build/release/bin/kudu as pid 12930
I20260925 14:49:42.291373 22327 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskY8Adyw/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790347486435597-22327-0/minicluster-data/master-2/wal/instance
I20260925 14:49:42.291568 22327 external_mini_cluster.cc:1428] Setting key 5c9ebd2f1e09b53eef5c6c6dad934c1f
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:42.296590 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.288011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.21.205.254:39057)
0925 14:49:42.288074 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.288076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.288276 (+ 200us) server_negotiation.cc:407] Connection header received
0925 14:49:42.288796 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.288797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.288810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.288867 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.289489 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.289977 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.290961 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.291184 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.291263 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.293517 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.293518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.293519 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.293527 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.294993 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.295511 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.295512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.295513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.295577 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.295882 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.295883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.295883 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.296000 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.296075 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.296364 (+ 289us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.296460 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 14:49:42.296733 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.288004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.254:39057 (local address 127.0.0.1:39098)
0925 14:49:42.288155 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.288191 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.288677 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.288940 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.288943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.289202 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.289393 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.289397 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.290091 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.290092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.290866 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.290868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.291268 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.291269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.291456 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.292028 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.292039 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.293418 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.295087 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.295093 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.295095 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.295403 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.295647 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.295648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.295648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.295771 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.296080 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.296087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.296224 (+ 137us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.296342 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.296489 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":73,"threads_started":1}
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:49:42.298048 13000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.300449 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.291946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.21.205.252:40459)
0925 14:49:42.292239 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.292243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.292258 (+ 15us) server_negotiation.cc:407] Connection header received
0925 14:49:42.292290 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.292291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.292306 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.292391 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.292982 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.293701 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.294473 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.294792 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.294891 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.297502 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.297507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.297508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.297514 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.299230 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.299582 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.299585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.299585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.299646 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.299878 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.299879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.299879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.300024 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.300207 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.300250 (+ 43us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.300335 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":82,"threads_started":1}
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:42.304937 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.298309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.253:33645 (local address 127.0.0.1:47930)
0925 14:49:42.298355 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.298365 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.298402 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.298533 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.298534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.298726 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.298766 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.298768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.299374 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.299375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.299909 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.299911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.300330 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.300331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.300411 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.300804 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.300808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.302207 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.303894 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.303895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.303896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.304064 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.304296 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.304297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.304297 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.304359 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.304621 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.304622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.304664 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.304752 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.304844 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260925 14:49:42.304937 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.298370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.21.205.253:33645)
0925 14:49:42.298417 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.298419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.298427 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:42.298452 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.298453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.298464 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.298537 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.298838 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.299274 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.299991 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.300244 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.300319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.302302 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.302303 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.302304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.302311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.303783 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.304161 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.304163 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.304163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.304220 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.304425 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.304427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.304427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.304550 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.304629 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.304778 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.304851 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.254@KRBTEST.COM
I20260925 14:49:42.305243 13000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } has no permanent_uuid. Determining permanent_uuid...
I20260925 14:49:42.305539 12848 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } attempt: 2
I20260925 14:49:42.308624 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.301251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39102 (local address 127.21.205.254:39057)
0925 14:49:42.301291 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.301293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.301357 (+ 64us) server_negotiation.cc:407] Connection header received
0925 14:49:42.301422 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.301423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.301430 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.301469 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.302006 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.302458 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.303125 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.303325 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.303399 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.305684 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.305685 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.305686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.305693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.307403 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.307808 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.307810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.307811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.307870 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.308092 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.308094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.308094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.308221 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.308411 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.308449 (+ 38us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.308536 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.252@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.254@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:49:42.312778 13002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.305505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36166)
0925 14:49:42.305543 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.305553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.305588 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.305722 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.305722 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.305869 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.305895 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.305896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.306819 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.306820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.307293 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.307294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.307773 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.307774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.307861 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.308745 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.308751 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.310113 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.311744 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.311744 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.311745 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.311986 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.312201 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.312202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.312202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.312279 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.312521 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.312521 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.312552 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.312592 (+ 40us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.312682 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260925 14:49:42.313115 13004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.305548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.21.205.252:40459)
0925 14:49:42.305593 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.305595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.305605 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:49:42.305633 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.305634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.305645 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.305690 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.305974 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.306742 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.307374 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.307690 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.307763 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.310188 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.310189 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.310189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.310196 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.311624 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.312059 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.312060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.312060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.312109 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.312347 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.312348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.312348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.312458 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.312894 (+ 436us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.312935 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.313043 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:49:42.314478 12851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.305760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36172)
0925 14:49:42.305808 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.305817 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.305852 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.306402 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.306403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.306555 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.306588 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.306589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.307533 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.307534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.308330 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.308332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.308925 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.308926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.308990 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.309397 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.309402 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.310881 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.313446 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.313447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.313448 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.313630 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.313850 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.313851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.313851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.313910 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.314214 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.314215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.314259 (+ 44us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.314303 (+ 44us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.314395 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347781, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.205.253@KRBTEST.COM
I20260925 14:49:42.314922 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.305928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36172 (local address 127.21.205.252:40459)
0925 14:49:42.306260 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.306262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.306270 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:42.306296 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.306297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.306306 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.306373 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.306687 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.307437 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.308414 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.308832 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.308899 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.310971 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.310972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.310973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.310979 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.313369 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.313706 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.313708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.313708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.313771 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.313980 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.313981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.313981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.314130 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.314682 (+ 552us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.314724 (+ 42us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.314805 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":125,"threads_started":1}
I20260925 14:49:42.315140 13000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: Bootstrap starting.
I20260925 14:49:42.316006 13000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:42.316346 13000 log.cc:824] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:42.316560 12848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: Bootstrap starting.
I20260925 14:49:42.317111 13000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: No bootstrap required, opened a new log
I20260925 14:49:42.317286 12848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:42.317633 12848 log.cc:824] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:42.318342 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.309831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47932 (local address 127.21.205.253:33645)
0925 14:49:42.309889 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.309891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.309898 (+ 7us) server_negotiation.cc:407] Connection header received
0925 14:49:42.309920 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.309921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.309929 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.309987 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.310588 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.311306 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.312314 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.312632 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.312694 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.315146 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.315147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.315147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.315154 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.316983 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.317481 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.317482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.317483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.317540 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.317788 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.317789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.317789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.317928 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.318137 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.318174 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.318249 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 14:49:42.318655 12848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: No bootstrap required, opened a new log
I20260925 14:49:42.319169 13000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.319375 13000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5889780c6887422a820e98a5c2cd7c0a, State: Initialized, Role: FOLLOWER
I20260925 14:49:42.319479 13000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.319871 13000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:42.319984 22327 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 14:49:42.319917 13008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.320178 13008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.320945 12848 raft_consensus.cc:348] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.320995 12822 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "c82c0b5a9e404f36a5609303602b9be7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39102
I20260925 14:49:42.321092 12848 raft_consensus.cc:729] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c82c0b5a9e404f36a5609303602b9be7, State: Initialized, Role: FOLLOWER
I20260925 14:49:42.321168 12848 consensus_queue.cc:260] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.321476 12848 sys_catalog.cc:564] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:42.321584 12822 raft_consensus.cc:482] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 14:49:42.321643 12822 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 0 FOLLOWER]: Advancing to term 1
W20260925 14:49:42.322063 13019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:42.322100 13019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:42.322587 12822 raft_consensus.cc:504] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.323272 12973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c82c0b5a9e404f36a5609303602b9be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5889780c6887422a820e98a5c2cd7c0a"
I20260925 14:49:42.323390 12973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:42.324434 12973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c82c0b5a9e404f36a5609303602b9be7 in term 1.
I20260925 14:49:42.324708 12799 leader_election.cc:302] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5889780c6887422a820e98a5c2cd7c0a, c82c0b5a9e404f36a5609303602b9be7; no voters:
I20260925 14:49:42.322844 12822 leader_election.cc:288] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [CANDIDATE]: Term 1 election: Requested vote from peers a38ad9a475144c61a38ffa188889c711 (127.21.205.253:33645), 5889780c6887422a820e98a5c2cd7c0a (127.21.205.252:40459)
I20260925 14:49:42.321961 12930 master_runner.cc:186] Error getting master registration for 127.21.205.254:39057: OK, instance_id { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" instance_seqno: 1790347781963469 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:49:42.325279 12930 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 14:49:42.323387 13032 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:42.326082 13032 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:42.327911 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.328200 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.328125 13020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 14:49:42.328356 13020 raft_consensus.cc:686] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [term 1 LEADER]: Becoming Leader. State: Replica: c82c0b5a9e404f36a5609303602b9be7, State: Running, Role: LEADER
I20260925 14:49:42.328490 13020 consensus_queue.cc:237] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.326689 12780 master_runner.cc:186] Error getting master registration for 127.21.205.253:33645: OK, instance_id { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" instance_seqno: 1790347782110654 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 14:49:42.329746 12780 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 14:49:42.331156 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c82c0b5a9e404f36a5609303602b9be7. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.331350 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:42.335386 12850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.325934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.21.205.254:39057)
0925 14:49:42.325981 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.325984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.326752 (+ 768us) server_negotiation.cc:407] Connection header received
0925 14:49:42.326838 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.326858 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.326871 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.326937 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.327689 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.328465 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.329588 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.329889 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.329971 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.331959 (+ 1988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.331960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.331961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.331973 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.333967 (+ 1994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.334548 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.334549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.334550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.334615 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.334824 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.334838 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.334838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.334977 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.335098 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.335214 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.335293 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260925 14:49:42.343895 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.326622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.21.205.252:40459)
0925 14:49:42.326669 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.326671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.335692 (+ 9021us) server_negotiation.cc:407] Connection header received
0925 14:49:42.335888 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.335889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.335903 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.335975 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.336576 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.337273 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.338408 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.338778 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.338862 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.340669 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.340670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.340671 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.340679 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.342422 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.342858 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.342859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.342860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.342917 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.343153 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.343154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.343154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.343283 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.343378 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.343695 (+ 317us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.343774 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260925 14:49:42.343919 13043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 14:49:42.344442 13043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 14:49:42.344463 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.326648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.21.205.253:33645)
0925 14:49:42.326696 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.326699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.332620 (+ 5921us) server_negotiation.cc:407] Connection header received
0925 14:49:42.332726 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.332727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.332740 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.332770 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.333417 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.334178 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.337152 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.337460 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.337502 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.340430 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.340431 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.340432 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.340439 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.342501 (+ 2062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.343101 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.343102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.343103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.343158 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.343453 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.343453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.343454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.343593 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.343669 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.344274 (+ 605us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.344370 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260925 14:49:42.346240 12973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [term 1 FOLLOWER]: Refusing update from remote peer c82c0b5a9e404f36a5609303602b9be7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:42.346556 12923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } attempt: 3
I20260925 14:49:42.346583 13034 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 14:49:42.348116 13008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c82c0b5a9e404f36a5609303602b9be7. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.348230 13008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.350737 13034 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.350828 13034 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:42.352032 13008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.352113 13008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.352447 13044 mvcc.cc:205] Tried to move back new op lower bound from 7333264516487589888 to 7333264516425678848. Current Snapshot: MvccSnapshot[applied={T|T < 7333264516487589888}]
I20260925 14:49:42.352680 13045 mvcc.cc:205] Tried to move back new op lower bound from 7333264516487589888 to 7333264516425678848. Current Snapshot: MvccSnapshot[applied={T|T < 7333264516487589888}]
Sep 25 14:49:42 dist-test-slave-d7lw krb5kdc[12766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790347782, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.205.253@KRBTEST.COM for kudu/127.21.205.252@KRBTEST.COM
I20260925 14:49:42.356264 13043 catalog_manager.cc:1347] Generated new cluster ID: 9db6bfdf337245f1bcf00389fecbc27d
I20260925 14:49:42.356328 13043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 14:49:42.356419 13005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.346963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.21.205.252:40459)
0925 14:49:42.347028 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.347030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.347040 (+ 10us) server_negotiation.cc:407] Connection header received
0925 14:49:42.347062 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.347063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.347075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.347137 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.348592 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.349355 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.350357 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.350670 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.350729 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.353308 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 14:49:42.353310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 14:49:42.353310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 14:49:42.353317 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0925 14:49:42.354915 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.355296 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.355297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.355297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.355367 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 14:49:42.355593 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 14:49:42.355594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 14:49:42.355595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 14:49:42.355726 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 14:49:42.355860 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:42.356250 (+ 390us) server_negotiation.cc:299] Negotiation successful
0925 14:49:42.356335 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:42.356582 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:42.346781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.205.252:40459 (local address 127.0.0.1:36180)
0925 14:49:42.346855 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0925 14:49:42.346865 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0925 14:49:42.346899 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 14:49:42.348298 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 14:49:42.348299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 14:49:42.348458 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0925 14:49:42.348501 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.348503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.349441 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.349442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.350262 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 14:49:42.350264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 14:49:42.350884 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 14:49:42.350886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 14:49:42.350961 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:42.351470 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 14:49:42.351475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0925 14:49:42.353214 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 14:49:42.355008 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.355009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.355010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.355221 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.355451 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 14:49:42.355452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 14:49:42.355452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0925 14:49:42.355515 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 14:49:42.355817 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 14:49:42.355818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 14:49:42.355863 (+ 45us) client_negotiation.cc:769] Sending connection context
0925 14:49:42.355934 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 14:49:42.356037 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 14:49:42.358004 12923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: Bootstrap starting.
I20260925 14:49:42.358736 12923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: Neither blocks nor log segments found. Creating new log.
I20260925 14:49:42.359019 12923 log.cc:824] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: Log is configured to *not* fsync() on all Append() calls
I20260925 14:49:42.359876 12923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: No bootstrap required, opened a new log
I20260925 14:49:42.361894 12923 raft_consensus.cc:348] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.362044 12923 raft_consensus.cc:729] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a38ad9a475144c61a38ffa188889c711, State: Initialized, Role: FOLLOWER
I20260925 14:49:42.362125 12923 consensus_queue.cc:260] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } }
I20260925 14:49:42.362421 12923 sys_catalog.cc:564] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 14:49:42.362560 12896 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [term 0 FOLLOWER]: Advancing to term 1
I20260925 14:49:42.363454 12896 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [term 1 FOLLOWER]: Refusing update from remote peer c82c0b5a9e404f36a5609303602b9be7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 14:49:42.363762 13034 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 14:49:42.364744 13062 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 14:49:42.364797 13062 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 14:49:42.364147 12897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c82c0b5a9e404f36a5609303602b9be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a38ad9a475144c61a38ffa188889c711"
I20260925 14:49:42.365569 13050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.365671 13050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.365736 13050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c82c0b5a9e404f36a5609303602b9be7. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.365784 13050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.365823 13050 sys_catalog.cc:455] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.365885 13050 sys_catalog.cc:458] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 14:49:42.370661 13065 mvcc.cc:205] Tried to move back new op lower bound from 7333264516487589888 to 7333264516425678848. Current Snapshot: MvccSnapshot[applied={T|T < 7333264516487589888}]
I20260925 14:49:42.371644 12897 raft_consensus.cc:2396] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c82c0b5a9e404f36a5609303602b9be7 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260925 14:49:42.373893 13020 sys_catalog.cc:455] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c82c0b5a9e404f36a5609303602b9be7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c82c0b5a9e404f36a5609303602b9be7" member_type: VOTER last_known_addr { host: "127.21.205.254" port: 39057 } } peers { permanent_uuid: "a38ad9a475144c61a38ffa188889c711" member_type: VOTER last_known_addr { host: "127.21.205.253" port: 33645 } } peers { permanent_uuid: "5889780c6887422a820e98a5c2cd7c0a" member_type: VOTER last_known_addr { host: "127.21.205.252" port: 40459 } } }
I20260925 14:49:42.374001 13020 sys_catalog.cc:458] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7 [sys.catalog]: This master's current role is: LEADER
I20260925 14:49:42.375144 13043 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 14:49:42.375907 13043 catalog_manager.cc:1504] Loading token signing keys...
I20260925 14:49:42.398550 13043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c82c0b5a9e404f36a5609303602b9be7: Generated new TSK 0
I20260925 14:49:42.398856 13043 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 14:49:42.424137 12850 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.422395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39116 (local address 127.21.205.254:39057)
0925 14:49:42.422443 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.422445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.423414 (+ 969us) server_negotiation.cc:407] Connection header received
0925 14:49:42.423536 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.423538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.423550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.423583 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.424063 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39116: BlockingRecv error: recv got EOF from 127.0.0.1:39116 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 14:49:42.424888 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.422825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.21.205.253:33645)
0925 14:49:42.422869 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.422871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.424377 (+ 1506us) server_negotiation.cc:407] Connection header received
0925 14:49:42.424473 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.424474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.424485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.424514 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.424828 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47956: BlockingRecv error: recv got EOF from 127.0.0.1:47956 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260925 14:49:42.425577 13005 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.423096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.21.205.252:40459)
0925 14:49:42.423144 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.423147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.425103 (+ 1956us) server_negotiation.cc:407] Connection header received
0925 14:49:42.425183 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.425184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.425194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.425222 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.425504 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36190: BlockingRecv error: recv got EOF from 127.0.0.1:36190 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260925 14:49:42.429836 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.429056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47962 (local address 127.21.205.253:33645)
0925 14:49:42.429090 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.429092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.429293 (+ 201us) server_negotiation.cc:407] Connection header received
0925 14:49:42.429399 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.429400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.429418 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.429446 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.429774 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47962: BlockingRecv error: recv got EOF from 127.0.0.1:47962 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260925 14:49:42.432819 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.432054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47976 (local address 127.21.205.253:33645)
0925 14:49:42.432088 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.432090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.432341 (+ 251us) server_negotiation.cc:407] Connection header received
0925 14:49:42.432438 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.432439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.432449 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.432475 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.432776 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47976: BlockingRecv error: recv got EOF from 127.0.0.1:47976 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 14:49:42.436599 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.435845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47982 (local address 127.21.205.253:33645)
0925 14:49:42.435877 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.435879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.436108 (+ 229us) server_negotiation.cc:407] Connection header received
0925 14:49:42.436202 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.436202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.436212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.436239 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.436544 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47982: BlockingRecv error: recv got EOF from 127.0.0.1:47982 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260925 14:49:42.442380 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.441619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47998 (local address 127.21.205.253:33645)
0925 14:49:42.441654 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.441655 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.441885 (+ 230us) server_negotiation.cc:407] Connection header received
0925 14:49:42.441978 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.441979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.441988 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.442016 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.442334 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47998: BlockingRecv error: recv got EOF from 127.0.0.1:47998 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260925 14:49:42.452481 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.451583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.21.205.253:33645)
0925 14:49:42.451623 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.451625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.451918 (+ 293us) server_negotiation.cc:407] Connection header received
0925 14:49:42.452015 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.452016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.452038 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.452066 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.452431 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48010: BlockingRecv error: recv got EOF from 127.0.0.1:48010 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260925 14:49:42.470688 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.469730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48022 (local address 127.21.205.253:33645)
0925 14:49:42.469776 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.469778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.470090 (+ 312us) server_negotiation.cc:407] Connection header received
0925 14:49:42.470203 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.470205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.470217 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.470247 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.470631 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48022: BlockingRecv error: recv got EOF from 127.0.0.1:48022 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260925 14:49:42.505016 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.504008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48024 (local address 127.21.205.253:33645)
0925 14:49:42.504067 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.504069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.504391 (+ 322us) server_negotiation.cc:407] Connection header received
0925 14:49:42.504504 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.504506 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.504517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.504550 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.504956 (+ 406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48024: BlockingRecv error: recv got EOF from 127.0.0.1:48024 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260925 14:49:42.571187 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.570457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48038 (local address 127.21.205.253:33645)
0925 14:49:42.570551 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.570553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.570626 (+ 73us) server_negotiation.cc:407] Connection header received
0925 14:49:42.570692 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.570693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.570704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.570748 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.571129 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48038: BlockingRecv error: recv got EOF from 127.0.0.1:48038 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 14:49:42.701287 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.700555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.21.205.253:33645)
0925 14:49:42.700627 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.700630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.700716 (+ 86us) server_negotiation.cc:407] Connection header received
0925 14:49:42.700790 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.700791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.700801 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.700853 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.701209 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48042: BlockingRecv error: recv got EOF from 127.0.0.1:48042 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260925 14:49:42.959110 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 14:49:42.958490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48050 (local address 127.21.205.253:33645)
0925 14:49:42.958558 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:42.958560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:42.958571 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:42.958599 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:42.958599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:42.958609 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:42.958671 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0925 14:49:42.959002 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48050: BlockingRecv error: recv got EOF from 127.0.0.1:48050 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 14:49:43.322546 13019 catalog_manager.cc:1259] Loaded cluster ID: 9db6bfdf337245f1bcf00389fecbc27d
I20260925 14:49:43.322618 13019 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: loading cluster ID for follower catalog manager: success
I20260925 14:49:43.323469 13019 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: acquiring CA information for follower catalog manager: success
I20260925 14:49:43.323803 13019 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5889780c6887422a820e98a5c2cd7c0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:49:43.365196 13062 catalog_manager.cc:1259] Loaded cluster ID: 9db6bfdf337245f1bcf00389fecbc27d
I20260925 14:49:43.365276 13062 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: loading cluster ID for follower catalog manager: success
I20260925 14:49:43.366102 13062 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: acquiring CA information for follower catalog manager: success
I20260925 14:49:43.366411 13062 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a38ad9a475144c61a38ffa188889c711: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 14:49:43.475260 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:43.472509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.21.205.253:33645)
0925 14:49:43.472625 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:43.472628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:43.472706 (+ 78us) server_negotiation.cc:407] Connection header received
0925 14:49:43.472804 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:43.472805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:43.472870 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:43.472958 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:43.473120 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.473648 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.474421 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.474619 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.474687 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:43.474884 (+ 197us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:43.474993 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:43.475082 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:43.475104 (+ 22us) server_negotiation.cc:299] Negotiation successful
0925 14:49:43.475128 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":51,"threads_started":1}
I20260925 14:49:43.479243 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:43.477085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36192 (local address 127.21.205.252:40459)
0925 14:49:43.477184 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:43.477185 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:43.477193 (+ 8us) server_negotiation.cc:407] Connection header received
0925 14:49:43.477227 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:43.477228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:43.477266 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:43.477338 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:43.477403 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.477884 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.478559 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.478745 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.478794 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:43.478923 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:43.479014 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:43.479071 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:43.479107 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 14:49:43.479124 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
I20260925 14:49:43.483999 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:43.481039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.21.205.253:33645)
0925 14:49:43.481087 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:43.481089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:43.481191 (+ 102us) server_negotiation.cc:407] Connection header received
0925 14:49:43.481295 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:43.481296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:43.481308 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:43.481336 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:43.481529 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.482051 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483168 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.483470 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483544 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:43.483693 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:43.483725 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:43.483825 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:43.483888 (+ 63us) server_negotiation.cc:299] Negotiation successful
0925 14:49:43.483909 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260925 14:49:43.484025 13141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:43.481284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.21.205.252:40459)
0925 14:49:43.481322 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:43.481323 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:43.481332 (+ 9us) server_negotiation.cc:407] Connection header received
0925 14:49:43.481375 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:43.481376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:43.481385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:43.481452 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:43.481700 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.482336 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483079 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.483354 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483417 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:43.483593 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:43.483622 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:43.483674 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:43.483938 (+ 264us) server_negotiation.cc:299] Negotiation successful
0925 14:49:43.483961 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260925 14:49:43.484145 13147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 14:49:43.480880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.21.205.254:39057)
0925 14:49:43.481005 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0925 14:49:43.481007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0925 14:49:43.481018 (+ 11us) server_negotiation.cc:407] Connection header received
0925 14:49:43.481072 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 14:49:43.481073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 14:49:43.481128 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 14:49:43.481211 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 14:49:43.481380 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.482147 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483142 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 14:49:43.483367 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 14:49:43.483443 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 14:49:43.483668 (+ 225us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 14:49:43.483762 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 14:49:43.483837 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 14:49:43.483986 (+ 149us) server_negotiation.cc:299] Negotiation successful
0925 14:49:43.484007 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":64,"threads_started":1}
I20260925 14:49:43.485870 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12780
I20260925 14:49:43.493252 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12855
I20260925 14:49:43.498150 22327 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskY8Adyw/build/release/bin/kudu with pid 12930
2026-09-25T14:49:43Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2816 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5687 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297070 ms total)
[ PASSED ] 54 tests.
[ FAILED ] 2 tests, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
[ FAILED ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
2 FAILED TESTS